-- Logs begin at Fri 2022-05-13 13:10:52 UTC, end at Fri 2022-05-13 14:15:34 UTC. -- May 13 13:39:30.249519 dv-cinder-841753-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 13 13:39:30.259912 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 13 13:39:30.264072 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 13 13:39:30 2022] *** May 13 13:39:30.264393 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 13 13:39:30.264555 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 13 13:39:30.264694 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: nodename: dv-cinder-841753-1-iscsi May 13 13:39:30.264833 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: machine: x86_64 May 13 13:39:30.264981 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: clock source: unix May 13 13:39:30.265117 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: pcre jit disabled May 13 13:39:30.265252 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: detected number of CPU cores: 4 May 13 13:39:30.265415 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: current working directory: / May 13 13:39:30.265629 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: detected binary path: /usr/bin/uwsgi-core May 13 13:39:30.265823 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: your processes number limit is 47680 May 13 13:39:30.266018 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: your memory page size is 4096 bytes May 13 13:39:30.266239 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: detected max file descriptor number: 2048 May 13 13:39:30.266411 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: lock engine: pthread robust mutexes May 13 13:39:30.266856 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: thunder lock: enabled May 13 13:39:30.267275 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 13 13:39:30.267498 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 13 13:39:30.314151 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: Python main interpreter initialized at 0x55f13a2c89c0 May 13 13:39:30.314151 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: python threads support enabled May 13 13:39:30.314151 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: your server socket listen backlog is limited to 100 connections May 13 13:39:30.314151 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: your mercy for graceful operations on workers is 90 seconds May 13 13:39:30.314550 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: mapped 403077 bytes (393 KB) for 2 cores May 13 13:39:30.314622 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: *** Operational MODE: preforking *** May 13 13:39:30.314933 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: *** uWSGI is running in multiple interpreter mode *** May 13 13:39:30.314933 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: spawned uWSGI master process (pid: 86347) May 13 13:39:30.315280 dv-cinder-841753-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 13 13:39:30.316424 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: spawned uWSGI worker 1 (pid: 86362, cores: 1) May 13 13:39:30.316424 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: spawned uWSGI worker 2 (pid: 86363, cores: 1) May 13 13:39:30.316424 dv-cinder-841753-1-iscsi devstack@keystone.service[86347]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 13 13:39:36.667605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 13:39:36.667605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: from cryptography.utils import int_from_bytes May 13 13:39:36.668418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 13:39:36.668418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: from cryptography.utils import int_from_bytes May 13 13:39:36.721916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 13 13:39:36.722131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 13 13:39:36.722353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] command line args: [] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 13 13:39:36.722552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 13 13:39:36.722759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ================================================================================ {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 13 13:39:36.723009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] admin_token = **** {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.723255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] config_dir = [] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.723443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.723619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] config_source = [] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.723797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] debug = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.723997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.724366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.725550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.725550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.725550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.725550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] list_limit = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.725550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] log_config_append = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.725550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.725931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] log_dir = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.725931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] log_file = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.726418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.726731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.726998 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.727252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.727432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.727603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.727846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.728096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.728332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.728587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.728806 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.729662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.729885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.730117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.730378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.730901 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.731168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] publish_errors = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.731440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.731826 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.732175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.732304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.732547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.732796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] standard_threads = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.733007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.733251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.733432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.733602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] use_journal = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.733773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] use_json = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.733949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] use_stderr = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.734125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] use_syslog = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.734358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:36.734619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.734903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.735147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.735388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.735643 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.735878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.736127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.736363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.736612 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.736902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.737154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.737391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.737704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.737881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.738095 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.738344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.738683 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.738970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.739387 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.739756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.740179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.740523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.740803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.741266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.741580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.741803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.742137 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.742380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.742624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.742909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.743163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.743636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.743886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.744140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.744380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.744684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.744920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.745155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.745393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.745625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.745858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.746090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.746322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.746561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.746831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.747066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.747301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.747533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.747769 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.748005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.748255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.748488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.748765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.748969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.749202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.749439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.749675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.749907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.750159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.750396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.750638 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.750934 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.751144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.751540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.751885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.752310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.752588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.752873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.753160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.753669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.753932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.754179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.754422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.754737 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.755002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.755248 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.755492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.755741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.755990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.756227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.756467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.756707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.756947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.757227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.757475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.757840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.758133 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.758345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.758602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.758867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.759108 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.759346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.759582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.759870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.760049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.760341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.760587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.760829 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.761080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.761325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.761586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.761852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.762092 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.762328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] auth.external = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.762563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.762885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.763125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.763360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] auth.password = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.763594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] auth.token = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.763871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.764052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.764304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.764532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.764784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.765037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.765254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.765547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] credential.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.765734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.765991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.766240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.766470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.766742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.766968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.767236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.767492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.767729 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.767982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.768309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.768535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.768779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.769031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.769318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.769532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] federation.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.769771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.770062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.770266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.770472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.770804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.771019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.771228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.771436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.771687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.773476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.773476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.773476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.773476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.773476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.773476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.773476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.775083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.775083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.775083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.775083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.775083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.775083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.775083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.777525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.777525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.777525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.777525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.777525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.777525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.778777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.778777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.778777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.778777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.779453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.779453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.779898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.780119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.780476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.780955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.781273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.781663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.781959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.782207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.782467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.782740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.783000 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.783245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.783482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.783725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.783966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.784218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.784460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.784701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.784958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.785205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.785440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.785674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.785927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.786233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.786477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.786750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.786987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.787199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.787424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.787702 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.787923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.788174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.788409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.788778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.788993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.789205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.789346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.789559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.789987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.790300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.790559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.790839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.791091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.791379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.792160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.792395 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.792649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.792891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.793134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.793496 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.793713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.793925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.794130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.794354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.794561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.794848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.795076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.795282 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.795535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.795808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.796061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.796301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.796512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.796855 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.797080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] resource.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.797304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.797532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.797745 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.797950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.798176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.798376 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.798845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.798845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.799252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.799462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] role.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.799690 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] role.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.799900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.800109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.800320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.800531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.800739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.800964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.801192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.801397 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.801606 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.801854 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.802083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.802337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.802586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.802945 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.803300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.803527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.803824 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.804027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.804254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.804504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.804736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.804982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.805235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.805489 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.805721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.805941 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.806142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.806561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.806950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.807240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.807471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.807706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.807940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.808184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] token.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.808423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.808682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.809056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.809382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.809663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.809908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.810154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.810400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.810652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.810961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.811172 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.811399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.811614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.811849 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.812072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.812289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.812567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.812829 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.connection = **** {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.813062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.813311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.813531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.813756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.813997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.814234 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.814460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.814681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.814949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.815204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.815413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.815621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.815831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.816070 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.816327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.816557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.816794 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.817024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:36.817237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 13 13:39:36.854647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:36.855039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:36.855357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:36.868789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:36.900609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:36.901025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:36.961097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.024686 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.024955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.111632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.111858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.112191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.112485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.112767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.113036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.138584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.138861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.139203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.139531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.139813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.140093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.140391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.152922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.168298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.168547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.168880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.169163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.169440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.169780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.169990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.188091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.188341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 13:39:37.188773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.188979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 13:39:37.189376 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.189801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.189944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.191649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.204012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.204012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.204012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.205163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.213170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.216532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.216923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.217309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.217784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.218136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.218460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.218840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.219179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.219512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.219852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.220185 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.226282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 13:39:37.226282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: from cryptography.utils import int_from_bytes May 13 13:39:37.227019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 13:39:37.227019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: from cryptography.utils import int_from_bytes May 13 13:39:37.248061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.249836 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.250193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.255658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.255996 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.256350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.264477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.264763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.269597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.269886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.270314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.270653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.271020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.271378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.271707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.272040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.277002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 13 13:39:37.277228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 13 13:39:37.277382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] command line args: [] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 13 13:39:37.277670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 13 13:39:37.277849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ================================================================================ {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 13 13:39:37.278106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] admin_token = **** {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.278356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] config_dir = [] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.278548 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.278768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] config_source = [] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.278957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] debug = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.279181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.279425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.279605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.279861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.280119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.280351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] list_limit = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.280529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] log_config_append = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.280709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.280888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] log_dir = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.281066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] log_file = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.281308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.281672 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.281925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.282176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.282351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.282531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.282804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.283058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.283294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.283533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.283770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.283992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.284241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.284495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.284787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.285295 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.285477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] publish_errors = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.285713 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.285946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.286126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.286744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.286744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.287018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] standard_threads = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.287136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.287313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.287492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.287671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] use_journal = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.287855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] use_json = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.288042 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] use_stderr = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.288216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] use_syslog = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.288412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:39:37.288664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.288907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.289177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.289480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.289728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.290060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.290310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.290578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.290951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.291260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.291493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.291735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.291987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.292231 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.292512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.292766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.293010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.293272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.293533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.293791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.294034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.294356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.294633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.294982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.295238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.295549 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.295818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.296063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.296209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.296372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.296559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.296751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.296877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.297127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.297360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.297614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.297866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.298105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.298359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.298681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.298983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.299229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.299584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.299784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.300237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.300554 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.300833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.301124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.301340 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.301582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.301820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.302081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.302307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.302532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.302666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.302994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.303171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.303343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.303514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.303723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.303974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.304210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.304502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.304752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.304993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.305231 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.305504 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.305835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.306087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.306332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.306625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.307485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.307759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.308024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.308209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.308394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.308552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.308676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.308844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.309176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.309434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.309792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.309956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.310348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.310611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.310928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.311127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.311371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.311616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.311888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.312140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.312417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.312662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.312911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.313156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.313401 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.313641 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.313876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.314113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.314350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.314592 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.314885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.315185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.315324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.315447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.315626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.315868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.316220 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.316466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.316625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.316879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] auth.external = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.317084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.317385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.317624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.317861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] auth.password = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.318102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] auth.token = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.318343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.318614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.319460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.319460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.319852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.320045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.320230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.320230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.320578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] credential.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.320764 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.320949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.321202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.321449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.321700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.321944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.322216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.322477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.323310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.323578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.323876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.324182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.324582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.325003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.325340 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.325622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] federation.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.325821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.326017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.326232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.326425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.326617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.326863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.327074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.327074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.327458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.327652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.327846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.327846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.328211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.328402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.328606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.328736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.328857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.328857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.329081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.329471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.329471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.329696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.329902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.330139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.330385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.330641 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.330946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.331185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.331491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.331730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.331906 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.332897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.332897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.332897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.332897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.333553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.333553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.334173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.334173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.334173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.334523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.334523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.334948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.335082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.335827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.335827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.335827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.336477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.336477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.336477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.336816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.337061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.337308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.337558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.337802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.338045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.338880 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.339131 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.339458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.339796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.340056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.340309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.340558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.341004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.341140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.341297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.341547 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.341781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.342020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.342265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.342529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.342790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.343128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.343400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.343572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.344026 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.344026 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.344288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.344455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.344684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.344925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.345316 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.345527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.345964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.346408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.346676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.346937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.347212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.347541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.347752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.347995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.348234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.348514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.348768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.349013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.349267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.349586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.349908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.350282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.350436 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.350649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.350931 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.351176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.351418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.351656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.351901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] resource.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.352159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.352434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.352676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.352928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.353429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.353429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.353788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.355959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.355959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.355959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] role.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.355959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] role.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.355959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.357444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.357629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.357629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.357957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.358089 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.358209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.358330 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.358447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.358650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.358650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.358650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.358650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.358650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.358650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.358650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.359792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.359792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.359792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.359792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.360224 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.360352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.360533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.360787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.361028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.361288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.361544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.361787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.362026 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.362264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.362503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.362782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.363045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.363298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.363571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.363782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.364016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.364259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] token.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.364496 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.364746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.364993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.365252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.365503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.365745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.365990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.366236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.366616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.366811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.367250 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.367518 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.367518 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.367768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.367884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.368155 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.368311 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.368540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.369197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.connection = **** {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.369197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.370683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.370683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.370683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.370683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.371209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.371408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.371537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.371722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.371722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.371722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.372390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.372390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.372390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.372897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.372897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.373166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.373284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.373517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.373832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:39:37.373972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 13 13:39:37.375090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.375444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.375806 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.376219 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.376568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.391144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.392653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.392653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.392653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.392653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.392653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.393732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.402885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.403135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.403330 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.411617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.412348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.415181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.418665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.420338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.422027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.422363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.422740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.430255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.431239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.431474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.432624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.435966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.436321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.436667 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.437004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.446512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.446902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.447497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.456856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.457238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.457711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.457887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.458227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.458575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.459119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.459356 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.459594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.460034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.460386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.460823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.461178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.461482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.476990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.506197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f13a2c89c0 pid: 86363 (default app) May 13 13:39:37.510111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582b0a3-3b41-43fa-8490-5a6b4e5c3818 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:37.511104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582b0a3-3b41-43fa-8490-5a6b4e5c3818 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:37.511503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582b0a3-3b41-43fa-8490-5a6b4e5c3818 None None] PATH_INFO: `/v3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:37.516559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1/1] 10.222.0.79 () {50 vars in 791 bytes} [Fri May 13 13:39:37 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:39:37.516775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.516775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.568919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.569182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.569375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.569599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.569818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.570047 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.594932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.595282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.595624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.595938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.596202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.596455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.596815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.611086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.615430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.615781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.616125 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.616455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.616777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.617103 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.617423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:39:37.634620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.634912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 13:39:37.635822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.636105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 13:39:37.636514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.636917 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.637314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.637707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.661183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.661183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.661183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.662169 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.674944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.677952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.678252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.678805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.678975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.680485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.681140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.681722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.682254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.683967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.684502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.684989 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.723075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.725228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.725454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.731877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.732089 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.732476 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.742150 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.742752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.751525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.751970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.752425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.752884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.753268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.753660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.754098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.754558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.778974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.779292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.784845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.789945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.790290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.801210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.805257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.809548 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.809915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.810356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.810835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.811433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.811778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.833760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.834209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.834609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.835038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.844061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.846164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.846446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.849730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.850010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.850312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.850593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.850904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.865568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.865874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.866123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.866404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.866684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.866983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.867274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.890210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.893599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.897082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.899350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.901689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.901994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.902324 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.915119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.918234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.923545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.923952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.924341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.924731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.940971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.941491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.941981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.953015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:39:37.953376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.953593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.953880 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.954407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.954702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.954997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.955487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.955807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.956264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.956902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.957107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.957649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:37.958274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:39:38.009979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x55f13a2c89c0 pid: 86362 (default app) May 13 13:39:46.300736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c0d1d-f5d2-4953-acef-fed65abaf5cb None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:46.301423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c0d1d-f5d2-4953-acef-fed65abaf5cb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:46.301423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c0d1d-f5d2-4953-acef-fed65abaf5cb None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:46.302548 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 2/2] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:39:46.313795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b4ac6-1595-43ab-b824-96d90ed16bca None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:46.314550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b4ac6-1595-43ab-b824-96d90ed16bca None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:46.314760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b4ac6-1595-43ab-b824-96d90ed16bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:46.350169 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG oslo_db.sqlalchemy.engines [None req-1d5b4ac6-1595-43ab-b824-96d90ed16bca None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86362) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 13 13:39:46.617414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG passlib.handlers.bcrypt [None req-1d5b4ac6-1595-43ab-b824-96d90ed16bca None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86362) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 13 13:39:46.617851 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG passlib.handlers.bcrypt [None req-1d5b4ac6-1595-43ab-b824-96d90ed16bca None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86362) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 13 13:39:46.685446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1d5b4ac6-1595-43ab-b824-96d90ed16bca None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:39:46.687912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1d5b4ac6-1595-43ab-b824-96d90ed16bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:39:46.893309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1/3] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:39:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 585 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 13 13:39:47.050493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c0d1d-f5d2-4953-acef-fed65abaf5cb None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:47.105738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG oslo_db.sqlalchemy.engines [None req-a23c0d1d-f5d2-4953-acef-fed65abaf5cb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86363) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 13 13:39:47.248561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c0d1d-f5d2-4953-acef-fed65abaf5cb None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:47.253235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d255bff-d1e1-458c-bf85-711afe7aeffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:47.253918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d255bff-d1e1-458c-bf85-711afe7aeffb admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:47.254200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d255bff-d1e1-458c-bf85-711afe7aeffb admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:47.254477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d255bff-d1e1-458c-bf85-711afe7aeffb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:47.265250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d255bff-d1e1-458c-bf85-711afe7aeffb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:47.320181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8d255bff-d1e1-458c-bf85-711afe7aeffb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 13 13:39:47.321142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d255bff-d1e1-458c-bf85-711afe7aeffb admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:47.332139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 3/4] 10.222.0.79 () {58 vars in 1324 bytes} [Fri May 13 13:39:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:39:47.339913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b4ac6-1595-43ab-b824-96d90ed16bca None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:47.359870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b4ac6-1595-43ab-b824-96d90ed16bca None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:47.367342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86757c-aea4-4b32-8cc2-35f3e1f54102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:47.369398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86757c-aea4-4b32-8cc2-35f3e1f54102 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:47.370448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86757c-aea4-4b32-8cc2-35f3e1f54102 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:47.370987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86757c-aea4-4b32-8cc2-35f3e1f54102 admin admin] PATH_INFO: `/v3/projects/584c523555e7486e977f27ddcc51047b` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:47.373981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86757c-aea4-4b32-8cc2-35f3e1f54102 admin admin] RBAC: Authorizing `identity:get_project(project_id=584c523555e7486e977f27ddcc51047b)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:47.483712 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9d86757c-aea4-4b32-8cc2-35f3e1f54102 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 13 13:39:47.484310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86757c-aea4-4b32-8cc2-35f3e1f54102 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:47.494344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 2/5] 10.222.0.79 () {56 vars in 1207 bytes} [Fri May 13 13:39:47 2022] GET /identity/v3/projects/584c523555e7486e977f27ddcc51047b => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:39:50.336152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a08d0b-f7aa-43cf-ab65-47d33513a22e None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:50.336692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a08d0b-f7aa-43cf-ab65-47d33513a22e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:50.336830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a08d0b-f7aa-43cf-ab65-47d33513a22e None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:50.344261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 4/6] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:39:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:39:50.352459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0215e-f6f9-4a7b-81a5-f1d747ae4ac6 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:50.353013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0215e-f6f9-4a7b-81a5-f1d747ae4ac6 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:50.353254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0215e-f6f9-4a7b-81a5-f1d747ae4ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:50.380199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-80d0215e-f6f9-4a7b-81a5-f1d747ae4ac6 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:39:50.382263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-80d0215e-f6f9-4a7b-81a5-f1d747ae4ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:39:50.394770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 3/7] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:39:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 13 13:39:50.426297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a08d0b-f7aa-43cf-ab65-47d33513a22e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:50.443385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a08d0b-f7aa-43cf-ab65-47d33513a22e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:50.450738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ecc2c-5b8b-438d-a7a0-23fc1f4a216a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:50.451721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ecc2c-5b8b-438d-a7a0-23fc1f4a216a admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:50.452012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ecc2c-5b8b-438d-a7a0-23fc1f4a216a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:50.452291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ecc2c-5b8b-438d-a7a0-23fc1f4a216a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:50.464170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ecc2c-5b8b-438d-a7a0-23fc1f4a216a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:50.467253 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ecc2c-5b8b-438d-a7a0-23fc1f4a216a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:50.481100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 5/8] 10.222.0.79 () {58 vars in 1324 bytes} [Fri May 13 13:39:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:39:50.490844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0215e-f6f9-4a7b-81a5-f1d747ae4ac6 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:50.502161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0215e-f6f9-4a7b-81a5-f1d747ae4ac6 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:50.508671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b424d-66d3-4db7-b68e-27e526b4ccaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:50.510154 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b424d-66d3-4db7-b68e-27e526b4ccaf admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:50.510619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b424d-66d3-4db7-b68e-27e526b4ccaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:50.511126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b424d-66d3-4db7-b68e-27e526b4ccaf admin admin] PATH_INFO: `/v3/users/a0034eb1f585466fae466bc41ea6f933` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:50.512694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b424d-66d3-4db7-b68e-27e526b4ccaf admin admin] RBAC: Authorizing `identity:get_user(user_id=a0034eb1f585466fae466bc41ea6f933)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:50.514558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b424d-66d3-4db7-b68e-27e526b4ccaf admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:50.517278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 4/9] 10.222.0.79 () {56 vars in 1198 bytes} [Fri May 13 13:39:50 2022] GET /identity/v3/users/a0034eb1f585466fae466bc41ea6f933 => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:39:53.156214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91af640-ad29-4710-91a2-04377dfb742d None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:53.156799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91af640-ad29-4710-91a2-04377dfb742d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:53.156939 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91af640-ad29-4710-91a2-04377dfb742d None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:53.159042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 6/10] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:39:53.166870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85bf33-cc29-4ded-95ff-515eb24b4fd4 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:53.167628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85bf33-cc29-4ded-95ff-515eb24b4fd4 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:53.167628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85bf33-cc29-4ded-95ff-515eb24b4fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:53.191995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-5e85bf33-cc29-4ded-95ff-515eb24b4fd4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:39:53.193994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5e85bf33-cc29-4ded-95ff-515eb24b4fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:39:53.315635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 5/11] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:39:53.342820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91af640-ad29-4710-91a2-04377dfb742d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:53.417555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa657586-5070-4e1e-b522-d1921cf8b62e None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:53.417946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa657586-5070-4e1e-b522-d1921cf8b62e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:53.418202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa657586-5070-4e1e-b522-d1921cf8b62e None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:53.420003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 6/12] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:39:53.427235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0e0ea-9e32-4b74-aa20-9e6eb2390f2d None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:53.427567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0e0ea-9e32-4b74-aa20-9e6eb2390f2d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:53.428016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0e0ea-9e32-4b74-aa20-9e6eb2390f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:53.455346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-d7c0e0ea-9e32-4b74-aa20-9e6eb2390f2d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:39:53.457666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d7c0e0ea-9e32-4b74-aa20-9e6eb2390f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:39:53.547490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 7/13] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:39:53.569052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91af640-ad29-4710-91a2-04377dfb742d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:53.576236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0e0ea-9e32-4b74-aa20-9e6eb2390f2d None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:53.613501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445052f8-b3b4-4958-aab6-61f6cad76461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:53.614312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445052f8-b3b4-4958-aab6-61f6cad76461 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:53.614598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445052f8-b3b4-4958-aab6-61f6cad76461 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:53.614910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445052f8-b3b4-4958-aab6-61f6cad76461 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:53.631600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445052f8-b3b4-4958-aab6-61f6cad76461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-445052f8-b3b4-4958-aab6-61f6cad76461 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:39:53.644893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:39:53.647040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:39:53.648929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 7/14] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:39:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:39:53.655338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445052f8-b3b4-4958-aab6-61f6cad76461 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:53.728930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0e0ea-9e32-4b74-aa20-9e6eb2390f2d None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:53.774173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cfa8d-5154-478c-878f-acba90d196c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:53.774883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cfa8d-5154-478c-878f-acba90d196c0 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:53.775693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cfa8d-5154-478c-878f-acba90d196c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:53.775982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cfa8d-5154-478c-878f-acba90d196c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:53.797536 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445052f8-b3b4-4958-aab6-61f6cad76461 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:53.856295 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cfa8d-5154-478c-878f-acba90d196c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:53.857739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cfa8d-5154-478c-878f-acba90d196c0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:53.858493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f139c-c168-449e-aa17-e90f5c837aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:53.859346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f139c-c168-449e-aa17-e90f5c837aec None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:53.859727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f139c-c168-449e-aa17-e90f5c837aec None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:53.860214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f139c-c168-449e-aa17-e90f5c837aec None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:53.861382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f139c-c168-449e-aa17-e90f5c837aec None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:39:53.861939 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f139c-c168-449e-aa17-e90f5c837aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:53.864701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f139c-c168-449e-aa17-e90f5c837aec None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:53.889558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 8/15] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:39:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:39:53.897722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f139c-c168-449e-aa17-e90f5c837aec None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:53.969939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 8/16] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:39:53.978781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cfa8d-5154-478c-878f-acba90d196c0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:54.053640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f139c-c168-449e-aa17-e90f5c837aec None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:54.105971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cfa8d-5154-478c-878f-acba90d196c0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:54.110560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032221bd-ffbe-4e8a-8bfe-0303188bacce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:54.111404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032221bd-ffbe-4e8a-8bfe-0303188bacce None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:54.111678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032221bd-ffbe-4e8a-8bfe-0303188bacce None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:54.111950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032221bd-ffbe-4e8a-8bfe-0303188bacce None admin] PATH_INFO: `/v3/users/a0034eb1f585466fae466bc41ea6f933` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:54.113704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032221bd-ffbe-4e8a-8bfe-0303188bacce None admin] RBAC: Authorizing `identity:get_user(user_id=a0034eb1f585466fae466bc41ea6f933)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:54.115227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032221bd-ffbe-4e8a-8bfe-0303188bacce None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:54.116962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 9/17] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:39:53 2022] GET /identity/v3/users/a0034eb1f585466fae466bc41ea6f933 => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:39:54.122971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032221bd-ffbe-4e8a-8bfe-0303188bacce None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:54.149717 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4687b9f-ad09-4810-9c7f-96d31d22e3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:54.151135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4687b9f-ad09-4810-9c7f-96d31d22e3d3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:54.151432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4687b9f-ad09-4810-9c7f-96d31d22e3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:54.151681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4687b9f-ad09-4810-9c7f-96d31d22e3d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:54.162528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4687b9f-ad09-4810-9c7f-96d31d22e3d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:54.164581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4687b9f-ad09-4810-9c7f-96d31d22e3d3 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-f4687b9f-ad09-4810-9c7f-96d31d22e3d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:39:54.189703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:39:54.191152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:39:54.191948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:39:54.191948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:39:54.191948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:39:54.191948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:39:54.191948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 9/18] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:39:54.198915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4687b9f-ad09-4810-9c7f-96d31d22e3d3 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:54.238504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032221bd-ffbe-4e8a-8bfe-0303188bacce None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:54.274683 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eafcfd-6cdb-43d8-86b2-3f71fb3d0deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:54.276060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eafcfd-6cdb-43d8-86b2-3f71fb3d0deb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:54.276375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eafcfd-6cdb-43d8-86b2-3f71fb3d0deb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:54.276609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eafcfd-6cdb-43d8-86b2-3f71fb3d0deb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:54.279365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eafcfd-6cdb-43d8-86b2-3f71fb3d0deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:54.279503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eafcfd-6cdb-43d8-86b2-3f71fb3d0deb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:54.281114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 10/19] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:39:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:39:54.287186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eafcfd-6cdb-43d8-86b2-3f71fb3d0deb None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:54.310830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4687b9f-ad09-4810-9c7f-96d31d22e3d3 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:54.349820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e6eea-b9d9-41e7-a42b-2275c5eb99a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:54.355049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e6eea-b9d9-41e7-a42b-2275c5eb99a6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:54.355404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e6eea-b9d9-41e7-a42b-2275c5eb99a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:54.355690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e6eea-b9d9-41e7-a42b-2275c5eb99a6 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:54.356650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e6eea-b9d9-41e7-a42b-2275c5eb99a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:39:54.357028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e6eea-b9d9-41e7-a42b-2275c5eb99a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:54.358860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e6eea-b9d9-41e7-a42b-2275c5eb99a6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:54.379490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 10/20] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:39:54.517667 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eafcfd-6cdb-43d8-86b2-3f71fb3d0deb None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:54.562092 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64828e-9d9a-4a46-a822-66ff095bb552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:54.562784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64828e-9d9a-4a46-a822-66ff095bb552 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:54.563072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64828e-9d9a-4a46-a822-66ff095bb552 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:54.563353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64828e-9d9a-4a46-a822-66ff095bb552 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:54.564197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64828e-9d9a-4a46-a822-66ff095bb552 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.domain.id=default, user.id=a0034eb1f585466fae466bc41ea6f933) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:39:54.564515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64828e-9d9a-4a46-a822-66ff095bb552 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:54.565986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64828e-9d9a-4a46-a822-66ff095bb552 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:54.575527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 11/21] 10.222.0.79 () {56 vars in 1322 bytes} [Fri May 13 13:39:54 2022] GET /identity/v3/role_assignments?user.id=a0034eb1f585466fae466bc41ea6f933&scope.domain.id=default&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 233 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:39:57.587461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb449-4529-4dce-9ab8-363e13055d49 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:57.589299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb449-4529-4dce-9ab8-363e13055d49 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:57.589804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eb449-4529-4dce-9ab8-363e13055d49 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:57.591826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 11/22] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:39:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:39:57.597811 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709aeee-80ae-4e9e-98b8-d2f0034b8bd6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:57.598754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709aeee-80ae-4e9e-98b8-d2f0034b8bd6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:57.599257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709aeee-80ae-4e9e-98b8-d2f0034b8bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:57.615289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG passlib.handlers.bcrypt [None req-5709aeee-80ae-4e9e-98b8-d2f0034b8bd6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86363) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 13 13:39:57.616036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG passlib.handlers.bcrypt [None req-5709aeee-80ae-4e9e-98b8-d2f0034b8bd6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86363) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 13 13:39:57.668132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-5709aeee-80ae-4e9e-98b8-d2f0034b8bd6 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:39:57.670142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5709aeee-80ae-4e9e-98b8-d2f0034b8bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:39:57.701844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e205c-8051-454b-a4da-669ce075f4f1 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:57.702798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e205c-8051-454b-a4da-669ce075f4f1 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:57.703498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e205c-8051-454b-a4da-669ce075f4f1 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:57.706110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 12/23] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:39:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:39:57.713274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f289e6-8a07-4085-9ad4-21b79db1439d None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:57.713955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f289e6-8a07-4085-9ad4-21b79db1439d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:57.714431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f289e6-8a07-4085-9ad4-21b79db1439d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:57.751774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-41f289e6-8a07-4085-9ad4-21b79db1439d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:39:57.753357 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-41f289e6-8a07-4085-9ad4-21b79db1439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:39:57.774849 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 12/24] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:39:57.795029 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709aeee-80ae-4e9e-98b8-d2f0034b8bd6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:57.863140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 13/25] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:39:57.917703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f289e6-8a07-4085-9ad4-21b79db1439d None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:57.958192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709aeee-80ae-4e9e-98b8-d2f0034b8bd6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:58.004636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7bb62-ac63-4010-becb-6bcd23d4c35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:58.006117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7bb62-ac63-4010-becb-6bcd23d4c35a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:58.006600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7bb62-ac63-4010-becb-6bcd23d4c35a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:58.007083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7bb62-ac63-4010-becb-6bcd23d4c35a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:58.026259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7bb62-ac63-4010-becb-6bcd23d4c35a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-55e7bb62-ac63-4010-becb-6bcd23d4c35a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:39:58.043348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:39:58.044402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:39:58.045259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:39:58.047251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 13/26] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:39:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:39:58.055347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eafadb-f2dc-4397-bf80-023b59bef2c2 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:58.055862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eafadb-f2dc-4397-bf80-023b59bef2c2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:58.056923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eafadb-f2dc-4397-bf80-023b59bef2c2 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:58.058919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 14/27] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:39:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:39:58.059867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eafadb-f2dc-4397-bf80-023b59bef2c2 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:58.076781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f289e6-8a07-4085-9ad4-21b79db1439d None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:58.122204 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebaee6-fe57-45c9-a349-44fe7dcf5674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:58.123364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebaee6-fe57-45c9-a349-44fe7dcf5674 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:58.123790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebaee6-fe57-45c9-a349-44fe7dcf5674 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:58.124204 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebaee6-fe57-45c9-a349-44fe7dcf5674 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:58.143126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebaee6-fe57-45c9-a349-44fe7dcf5674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:58.145628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebaee6-fe57-45c9-a349-44fe7dcf5674 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-a4ebaee6-fe57-45c9-a349-44fe7dcf5674 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:39:58.297743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:39:58.298823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:39:58.299933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:39:58.299933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:39:58.299933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:39:58.299933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:39:58.301823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 14/28] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:39:58.303425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f167a1-c28b-4d58-a3ad-941db8e787a1 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:58.304013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f167a1-c28b-4d58-a3ad-941db8e787a1 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:58.304662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f167a1-c28b-4d58-a3ad-941db8e787a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:58.335191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-14f167a1-c28b-4d58-a3ad-941db8e787a1 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:39:58.337063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-14f167a1-c28b-4d58-a3ad-941db8e787a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:39:58.357670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eafadb-f2dc-4397-bf80-023b59bef2c2 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:58.438386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 15/29] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:39:58.441498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f167a1-c28b-4d58-a3ad-941db8e787a1 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:58.442298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae41a40-87fb-4389-8fb0-423110a0a449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:58.443641 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae41a40-87fb-4389-8fb0-423110a0a449 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:58.446061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae41a40-87fb-4389-8fb0-423110a0a449 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:58.446502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae41a40-87fb-4389-8fb0-423110a0a449 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:58.449827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae41a40-87fb-4389-8fb0-423110a0a449 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:39:58.451731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae41a40-87fb-4389-8fb0-423110a0a449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:58.457612 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae41a40-87fb-4389-8fb0-423110a0a449 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:58.476760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 15/30] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:39:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:39:58.483293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae41a40-87fb-4389-8fb0-423110a0a449 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:58.560823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f167a1-c28b-4d58-a3ad-941db8e787a1 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:58.615883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0e72a-f09e-4bf8-8be1-a997038e677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:58.618162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0e72a-f09e-4bf8-8be1-a997038e677e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:58.618963 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0e72a-f09e-4bf8-8be1-a997038e677e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:58.619677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0e72a-f09e-4bf8-8be1-a997038e677e None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:58.621314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0e72a-f09e-4bf8-8be1-a997038e677e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:39:58.622094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0e72a-f09e-4bf8-8be1-a997038e677e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:58.625114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0e72a-f09e-4bf8-8be1-a997038e677e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:58.634311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae41a40-87fb-4389-8fb0-423110a0a449 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:58.647325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 16/31] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:39:58.649336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0e72a-f09e-4bf8-8be1-a997038e677e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:58.690974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cb9a4-3cef-42e8-bdd3-f3e539efc93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:58.692143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cb9a4-3cef-42e8-bdd3-f3e539efc93e None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:58.696333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cb9a4-3cef-42e8-bdd3-f3e539efc93e None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:58.698444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cb9a4-3cef-42e8-bdd3-f3e539efc93e None admin] PATH_INFO: `/v3/users/a0034eb1f585466fae466bc41ea6f933` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:58.703044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cb9a4-3cef-42e8-bdd3-f3e539efc93e None admin] RBAC: Authorizing `identity:get_user(user_id=a0034eb1f585466fae466bc41ea6f933)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:58.707676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cb9a4-3cef-42e8-bdd3-f3e539efc93e None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:58.711632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 16/32] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:39:58 2022] GET /identity/v3/users/a0034eb1f585466fae466bc41ea6f933 => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:39:58.715684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29285ea7-805c-4ada-ab57-ff84111d3a1c None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:58.716860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29285ea7-805c-4ada-ab57-ff84111d3a1c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:58.717654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29285ea7-805c-4ada-ab57-ff84111d3a1c None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:58.726877 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 17/33] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:39:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:39:58.726877 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29285ea7-805c-4ada-ab57-ff84111d3a1c None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:58.776509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0e72a-f09e-4bf8-8be1-a997038e677e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:58.814289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3922f6-b4cc-45e8-8c63-3d4d3abd6719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:58.815262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3922f6-b4cc-45e8-8c63-3d4d3abd6719 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:58.815690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3922f6-b4cc-45e8-8c63-3d4d3abd6719 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:58.816108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3922f6-b4cc-45e8-8c63-3d4d3abd6719 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:58.817074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3922f6-b4cc-45e8-8c63-3d4d3abd6719 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:58.818695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3922f6-b4cc-45e8-8c63-3d4d3abd6719 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:58.937342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29285ea7-805c-4ada-ab57-ff84111d3a1c None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:58.995253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 17/34] 10.222.0.79 () {60 vars in 1131 bytes} [Fri May 13 13:39:58 2022] POST /identity/v3/roles => generated 219 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:39:58.996530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3922f6-b4cc-45e8-8c63-3d4d3abd6719 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:59.008779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704d955-88dc-4703-be4a-0ffccbd22141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:59.010244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704d955-88dc-4703-be4a-0ffccbd22141 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.010605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704d955-88dc-4703-be4a-0ffccbd22141 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.010974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704d955-88dc-4703-be4a-0ffccbd22141 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.011778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704d955-88dc-4703-be4a-0ffccbd22141 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:59.013669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704d955-88dc-4703-be4a-0ffccbd22141 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:59.141041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3922f6-b4cc-45e8-8c63-3d4d3abd6719 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:59.184582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09babcb-8302-47ce-aeda-c7fb92c64cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:59.185837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09babcb-8302-47ce-aeda-c7fb92c64cb6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.186353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09babcb-8302-47ce-aeda-c7fb92c64cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.186918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09babcb-8302-47ce-aeda-c7fb92c64cb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.188626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09babcb-8302-47ce-aeda-c7fb92c64cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:59.190529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09babcb-8302-47ce-aeda-c7fb92c64cb6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:59.192996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 18/35] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:39:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:39:59.195140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f78bfc-9b36-4c20-a812-252c9083e497 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.195729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f78bfc-9b36-4c20-a812-252c9083e497 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.196240 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f78bfc-9b36-4c20-a812-252c9083e497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.216366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-93f78bfc-9b36-4c20-a812-252c9083e497 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:39:59.218351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-93f78bfc-9b36-4c20-a812-252c9083e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:39:59.219763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 18/36] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:39:58 2022] POST /identity/v3/projects => generated 301 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:39:59.221539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f162ae7-a17d-4290-b78e-2e64023bf9f9 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.221837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f162ae7-a17d-4290-b78e-2e64023bf9f9 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.222123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f162ae7-a17d-4290-b78e-2e64023bf9f9 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.223714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 19/37] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:39:59.224528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f162ae7-a17d-4290-b78e-2e64023bf9f9 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:59.332401 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 19/38] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:39:59.333078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b52b33-faf4-4605-a61f-0a59ce78a9ec None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.333483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b52b33-faf4-4605-a61f-0a59ce78a9ec None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.333958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b52b33-faf4-4605-a61f-0a59ce78a9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.381471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f162ae7-a17d-4290-b78e-2e64023bf9f9 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:59.384049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-d7b52b33-faf4-4605-a61f-0a59ce78a9ec None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:39:59.386271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d7b52b33-faf4-4605-a61f-0a59ce78a9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:39:59.438697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fff0e-a482-4cb8-b651-7a7d49062b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:59.439816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fff0e-a482-4cb8-b651-7a7d49062b22 None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.440220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fff0e-a482-4cb8-b651-7a7d49062b22 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.440549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fff0e-a482-4cb8-b651-7a7d49062b22 None admin] PATH_INFO: `/v3/domains/default/users/a0034eb1f585466fae466bc41ea6f933/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.443949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fff0e-a482-4cb8-b651-7a7d49062b22 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a0034eb1f585466fae466bc41ea6f933, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:59.447553 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fff0e-a482-4cb8-b651-7a7d49062b22 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:59.474442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 20/39] 10.222.0.79 () {58 vars in 1361 bytes} [Fri May 13 13:39:59 2022] PUT /identity/v3/domains/default/users/a0034eb1f585466fae466bc41ea6f933/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:39:59.477804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fff0e-a482-4cb8-b651-7a7d49062b22 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:59.519536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 20/40] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:39:59.521794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac5f7-6240-455e-a0a8-fce6eaa876f0 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.522547 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac5f7-6240-455e-a0a8-fce6eaa876f0 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.523060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac5f7-6240-455e-a0a8-fce6eaa876f0 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.525793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 21/41] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:39:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:39:59.534582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2fea8-a74a-41ac-b97f-8b5d8be8142e None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.535331 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2fea8-a74a-41ac-b97f-8b5d8be8142e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.536031 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2fea8-a74a-41ac-b97f-8b5d8be8142e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.561284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-16c2fea8-a74a-41ac-b97f-8b5d8be8142e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:39:59.563047 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-16c2fea8-a74a-41ac-b97f-8b5d8be8142e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:39:59.645937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fff0e-a482-4cb8-b651-7a7d49062b22 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:59.674057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 22/42] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:39:59.675729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2fea8-a74a-41ac-b97f-8b5d8be8142e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:59.711113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6796c-fd4f-4f87-a1b7-f0b6e81483d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:59.711956 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6796c-fd4f-4f87-a1b7-f0b6e81483d3 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.712969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6796c-fd4f-4f87-a1b7-f0b6e81483d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.713484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6796c-fd4f-4f87-a1b7-f0b6e81483d3 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.714691 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6796c-fd4f-4f87-a1b7-f0b6e81483d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:59.716989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6796c-fd4f-4f87-a1b7-f0b6e81483d3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:59.746304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 21/43] 10.222.0.79 () {60 vars in 1131 bytes} [Fri May 13 13:39:59 2022] POST /identity/v3/roles => generated 225 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:39:59.747647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6796c-fd4f-4f87-a1b7-f0b6e81483d3 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:39:59.833083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2fea8-a74a-41ac-b97f-8b5d8be8142e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:59.875246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174705d-ffef-4f1d-a300-455519a9d892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:59.876245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174705d-ffef-4f1d-a300-455519a9d892 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.876755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174705d-ffef-4f1d-a300-455519a9d892 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.877221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174705d-ffef-4f1d-a300-455519a9d892 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.878185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174705d-ffef-4f1d-a300-455519a9d892 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:59.879899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174705d-ffef-4f1d-a300-455519a9d892 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:59.903881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 23/44] 10.222.0.79 () {60 vars in 1131 bytes} [Fri May 13 13:39:59 2022] POST /identity/v3/roles => generated 223 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:39:59.942682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6796c-fd4f-4f87-a1b7-f0b6e81483d3 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:39:59.991093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec761309-692a-4be1-bc63-41fb36013bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:39:59.992149 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec761309-692a-4be1-bc63-41fb36013bac None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:39:59.992443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec761309-692a-4be1-bc63-41fb36013bac None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:39:59.992723 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec761309-692a-4be1-bc63-41fb36013bac None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:39:59.994841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec761309-692a-4be1-bc63-41fb36013bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:39:59.996554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec761309-692a-4be1-bc63-41fb36013bac None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:39:59.998924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 22/45] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:39:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:00.007695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174705d-ffef-4f1d-a300-455519a9d892 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:00.180032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174705d-ffef-4f1d-a300-455519a9d892 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:00.219581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a07da6-9449-4950-adc3-04c798970dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:00.220255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a07da6-9449-4950-adc3-04c798970dff None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:00.220536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a07da6-9449-4950-adc3-04c798970dff None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:00.220811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a07da6-9449-4950-adc3-04c798970dff None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:00.221546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a07da6-9449-4950-adc3-04c798970dff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:00.223069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a07da6-9449-4950-adc3-04c798970dff None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:00.253609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 24/46] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:40:00 2022] POST /identity/v3/projects => generated 312 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:40:02.384625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d5c8c-d328-464c-b68a-c475b6b4e3c0 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:02.385152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d5c8c-d328-464c-b68a-c475b6b4e3c0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:02.385152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d5c8c-d328-464c-b68a-c475b6b4e3c0 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:02.386515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 23/47] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:02.392774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77a890-b991-46a6-917e-3263f7e5d497 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:02.393790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77a890-b991-46a6-917e-3263f7e5d497 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:02.393790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77a890-b991-46a6-917e-3263f7e5d497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:02.411820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-0b77a890-b991-46a6-917e-3263f7e5d497 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:02.413051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0b77a890-b991-46a6-917e-3263f7e5d497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:02.499670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 25/48] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:02.523727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d5c8c-d328-464c-b68a-c475b6b4e3c0 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:02.666812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d5c8c-d328-464c-b68a-c475b6b4e3c0 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:02.717018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaf038-73d1-4b95-af21-f3fa38d30c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:02.717740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaf038-73d1-4b95-af21-f3fa38d30c12 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:02.717957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaf038-73d1-4b95-af21-f3fa38d30c12 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:02.718243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaf038-73d1-4b95-af21-f3fa38d30c12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:02.732084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaf038-73d1-4b95-af21-f3fa38d30c12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-edaaf038-73d1-4b95-af21-f3fa38d30c12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:02.741516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:02.743400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:02.744951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 24/49] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:40:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:02.752589 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77a890-b991-46a6-917e-3263f7e5d497 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:02.911674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77a890-b991-46a6-917e-3263f7e5d497 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:02.950974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a74fae-add8-44bb-b51f-40e78729e434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:02.951699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a74fae-add8-44bb-b51f-40e78729e434 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:02.951981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a74fae-add8-44bb-b51f-40e78729e434 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:02.952261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a74fae-add8-44bb-b51f-40e78729e434 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:02.953025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a74fae-add8-44bb-b51f-40e78729e434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:02.953334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a74fae-add8-44bb-b51f-40e78729e434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:02.955046 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a74fae-add8-44bb-b51f-40e78729e434 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:02.967670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 26/50] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:02.973387 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaf038-73d1-4b95-af21-f3fa38d30c12 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:03.132754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaf038-73d1-4b95-af21-f3fa38d30c12 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:03.191413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41237cb-a2ac-4a27-8a78-1f9d6514a5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:03.192925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41237cb-a2ac-4a27-8a78-1f9d6514a5bb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:03.193412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41237cb-a2ac-4a27-8a78-1f9d6514a5bb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:03.194566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41237cb-a2ac-4a27-8a78-1f9d6514a5bb None admin] PATH_INFO: `/v3/users/a0034eb1f585466fae466bc41ea6f933` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:03.196639 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41237cb-a2ac-4a27-8a78-1f9d6514a5bb None admin] RBAC: Authorizing `identity:get_user(user_id=a0034eb1f585466fae466bc41ea6f933)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:03.199434 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41237cb-a2ac-4a27-8a78-1f9d6514a5bb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:03.202609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 25/51] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:02 2022] GET /identity/v3/users/a0034eb1f585466fae466bc41ea6f933 => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:03.209116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a74fae-add8-44bb-b51f-40e78729e434 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:03.275717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b9411-ab60-46de-82b6-9da88d0b33ff None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:03.276097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b9411-ab60-46de-82b6-9da88d0b33ff None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:03.276432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b9411-ab60-46de-82b6-9da88d0b33ff None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:03.278309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 26/52] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:03.286285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67b4b2-1f13-4511-a0e5-4ae2f84f1a4b None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:03.286596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67b4b2-1f13-4511-a0e5-4ae2f84f1a4b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:03.287180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67b4b2-1f13-4511-a0e5-4ae2f84f1a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:03.307657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-cf67b4b2-1f13-4511-a0e5-4ae2f84f1a4b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:03.310348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-cf67b4b2-1f13-4511-a0e5-4ae2f84f1a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:03.326640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a74fae-add8-44bb-b51f-40e78729e434 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:03.366314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcf6f4-14f9-41f5-816f-015299aa9de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:03.367757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf6f4-14f9-41f5-816f-015299aa9de1 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:03.368244 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf6f4-14f9-41f5-816f-015299aa9de1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:03.368747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf6f4-14f9-41f5-816f-015299aa9de1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:03.370584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcf6f4-14f9-41f5-816f-015299aa9de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:03.373170 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcf6f4-14f9-41f5-816f-015299aa9de1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:03.375876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 27/53] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:03.382090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcf6f4-14f9-41f5-816f-015299aa9de1 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:03.424373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 27/54] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:03.457920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67b4b2-1f13-4511-a0e5-4ae2f84f1a4b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:03.503319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcf6f4-14f9-41f5-816f-015299aa9de1 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:03.540110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04e3de-7711-4743-9f5a-8f13b23b158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:03.541380 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04e3de-7711-4743-9f5a-8f13b23b158a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:03.541662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04e3de-7711-4743-9f5a-8f13b23b158a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:03.541940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04e3de-7711-4743-9f5a-8f13b23b158a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:03.542813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04e3de-7711-4743-9f5a-8f13b23b158a None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.domain.id=default, user.id=a0034eb1f585466fae466bc41ea6f933) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:03.543125 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04e3de-7711-4743-9f5a-8f13b23b158a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:03.544608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04e3de-7711-4743-9f5a-8f13b23b158a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:03.552261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 28/55] 10.222.0.79 () {56 vars in 1322 bytes} [Fri May 13 13:40:03 2022] GET /identity/v3/role_assignments?user.id=a0034eb1f585466fae466bc41ea6f933&scope.domain.id=default&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 530 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:03.621435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67b4b2-1f13-4511-a0e5-4ae2f84f1a4b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:03.688131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee176630-9520-4d3a-917d-b0cbaa2c0f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:03.689568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee176630-9520-4d3a-917d-b0cbaa2c0f0b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:03.690064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee176630-9520-4d3a-917d-b0cbaa2c0f0b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:03.690545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee176630-9520-4d3a-917d-b0cbaa2c0f0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:03.693036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee176630-9520-4d3a-917d-b0cbaa2c0f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:03.696027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee176630-9520-4d3a-917d-b0cbaa2c0f0b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:03.705688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 28/56] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:03.707119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04e3de-7711-4743-9f5a-8f13b23b158a None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:03.868205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04e3de-7711-4743-9f5a-8f13b23b158a None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:03.905429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065aa49-959e-4216-85d9-30d8306ab497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:03.906098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065aa49-959e-4216-85d9-30d8306ab497 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:03.906372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065aa49-959e-4216-85d9-30d8306ab497 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:03.906645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065aa49-959e-4216-85d9-30d8306ab497 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:03.907394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065aa49-959e-4216-85d9-30d8306ab497 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:03.908858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065aa49-959e-4216-85d9-30d8306ab497 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:03.938117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 29/57] 10.222.0.79 () {60 vars in 1140 bytes} [Fri May 13 13:40:03 2022] POST /identity/v3/projects => generated 298 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:40:06.293321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a5f51-1995-4a68-b983-c8c268061f35 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:06.293876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a5f51-1995-4a68-b983-c8c268061f35 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:06.293876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a5f51-1995-4a68-b983-c8c268061f35 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:06.295440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 29/58] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:06.301760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48663d-3e0f-46e5-a57c-d0cb66bbeedf None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:06.302340 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48663d-3e0f-46e5-a57c-d0cb66bbeedf None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:06.302340 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48663d-3e0f-46e5-a57c-d0cb66bbeedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:06.319707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-3f48663d-3e0f-46e5-a57c-d0cb66bbeedf None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:06.321186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-3f48663d-3e0f-46e5-a57c-d0cb66bbeedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:06.398939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 30/59] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:06.425680 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a5f51-1995-4a68-b983-c8c268061f35 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:06.535155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a5f51-1995-4a68-b983-c8c268061f35 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:06.570466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c590f0-d499-489e-b19a-3cdfebad0073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:06.571309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c590f0-d499-489e-b19a-3cdfebad0073 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:06.571644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c590f0-d499-489e-b19a-3cdfebad0073 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:06.571928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c590f0-d499-489e-b19a-3cdfebad0073 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:06.573205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c590f0-d499-489e-b19a-3cdfebad0073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:06.574671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c590f0-d499-489e-b19a-3cdfebad0073 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:06.576378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 30/60] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:06.582683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48663d-3e0f-46e5-a57c-d0cb66bbeedf None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:06.707160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48663d-3e0f-46e5-a57c-d0cb66bbeedf None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:06.743268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324669c8-416e-41ed-9aa7-7866b1afa629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:06.743988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324669c8-416e-41ed-9aa7-7866b1afa629 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:06.744270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324669c8-416e-41ed-9aa7-7866b1afa629 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:06.744544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324669c8-416e-41ed-9aa7-7866b1afa629 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:06.745274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324669c8-416e-41ed-9aa7-7866b1afa629 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:06.746775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324669c8-416e-41ed-9aa7-7866b1afa629 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:06.773307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 31/61] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:40:06 2022] POST /identity/v3/users => generated 275 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:40:11.754239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23414b7-b714-44fa-8d40-ef4c52eac969 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:11.755213 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23414b7-b714-44fa-8d40-ef4c52eac969 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:11.755651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23414b7-b714-44fa-8d40-ef4c52eac969 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:11.757586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 31/62] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:11.771574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d1de-eac1-46b4-a12d-928692cc072a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:11.772464 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d1de-eac1-46b4-a12d-928692cc072a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:11.772886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d1de-eac1-46b4-a12d-928692cc072a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:11.804433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5177-1273-43a9-bfcd-0c3c9bf5ea1a None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:11.804721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5177-1273-43a9-bfcd-0c3c9bf5ea1a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:11.805178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5177-1273-43a9-bfcd-0c3c9bf5ea1a None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:11.807164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 32/63] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:11.813375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836f2c8-436a-41cf-a824-9230117f2ab9 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:11.813920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836f2c8-436a-41cf-a824-9230117f2ab9 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:11.814382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836f2c8-436a-41cf-a824-9230117f2ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:11.818821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-23f9d1de-eac1-46b4-a12d-928692cc072a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:11.824693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-23f9d1de-eac1-46b4-a12d-928692cc072a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:11.872324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b836f2c8-436a-41cf-a824-9230117f2ab9 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:11.877789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b836f2c8-436a-41cf-a824-9230117f2ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:12.050812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 32/64] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:12.053766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2d31a-9bc9-4a5c-abdf-e4802f8ea495 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.054667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2d31a-9bc9-4a5c-abdf-e4802f8ea495 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.056500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2d31a-9bc9-4a5c-abdf-e4802f8ea495 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.059949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 33/65] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:12.072277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff16c4-8d7e-4852-9acb-ffef5d42f5a4 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.073250 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff16c4-8d7e-4852-9acb-ffef5d42f5a4 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.074077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff16c4-8d7e-4852-9acb-ffef5d42f5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.083696 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 33/66] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:12.104789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-58ff16c4-8d7e-4852-9acb-ffef5d42f5a4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:12.106442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836f2c8-436a-41cf-a824-9230117f2ab9 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:12.107783 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-58ff16c4-8d7e-4852-9acb-ffef5d42f5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:12.270790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 34/67] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:12.270790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff16c4-8d7e-4852-9acb-ffef5d42f5a4 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:12.293108 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836f2c8-436a-41cf-a824-9230117f2ab9 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:12.352790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854374f-9a3e-4b2f-8f58-741d87a843ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:12.354501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854374f-9a3e-4b2f-8f58-741d87a843ab None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.355382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854374f-9a3e-4b2f-8f58-741d87a843ab None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.356075 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854374f-9a3e-4b2f-8f58-741d87a843ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.375541 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854374f-9a3e-4b2f-8f58-741d87a843ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-2854374f-9a3e-4b2f-8f58-741d87a843ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:12.391256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:12.393232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:12.395302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:12.398498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 34/68] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:40:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:40:12.398832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7269a2-0042-4813-98f6-c33bd39dc90d None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.403226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7269a2-0042-4813-98f6-c33bd39dc90d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.403226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7269a2-0042-4813-98f6-c33bd39dc90d None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.403961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 35/69] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:12.405404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7269a2-0042-4813-98f6-c33bd39dc90d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:12.413819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff16c4-8d7e-4852-9acb-ffef5d42f5a4 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:12.471824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775df653-b0b6-4a3f-a01f-0de93400cb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:12.473090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df653-b0b6-4a3f-a01f-0de93400cb7e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.473543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df653-b0b6-4a3f-a01f-0de93400cb7e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.473981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df653-b0b6-4a3f-a01f-0de93400cb7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.492913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775df653-b0b6-4a3f-a01f-0de93400cb7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-775df653-b0b6-4a3f-a01f-0de93400cb7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:12.509775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:12.511219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:12.512447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 35/70] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:40:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:12.514589 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bd33e-a8c9-48b4-9ef1-a29a4074f06f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.514867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bd33e-a8c9-48b4-9ef1-a29a4074f06f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.515180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bd33e-a8c9-48b4-9ef1-a29a4074f06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.537494 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-fe6bd33e-a8c9-48b4-9ef1-a29a4074f06f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:12.539053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-fe6bd33e-a8c9-48b4-9ef1-a29a4074f06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:12.573443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7269a2-0042-4813-98f6-c33bd39dc90d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:12.626331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d06aac-db0c-4218-91bc-965a17af9d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:12.627614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d06aac-db0c-4218-91bc-965a17af9d86 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.628116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d06aac-db0c-4218-91bc-965a17af9d86 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.628601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d06aac-db0c-4218-91bc-965a17af9d86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.647636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d06aac-db0c-4218-91bc-965a17af9d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:12.670750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 36/71] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:12.670750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bd33e-a8c9-48b4-9ef1-a29a4074f06f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-05d06aac-db0c-4218-91bc-965a17af9d86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:12.671142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:12.672627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 36/72] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:12.674056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825a5c2-f141-4225-95d8-d34604addfe6 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.676750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825a5c2-f141-4225-95d8-d34604addfe6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.676750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825a5c2-f141-4225-95d8-d34604addfe6 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.676750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 37/73] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:12.676750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825a5c2-f141-4225-95d8-d34604addfe6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:12.802818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825a5c2-f141-4225-95d8-d34604addfe6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:12.843853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ff89d-4b7e-4066-9ac6-f09179f8727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:12.844136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bd33e-a8c9-48b4-9ef1-a29a4074f06f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:12.844791 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ff89d-4b7e-4066-9ac6-f09179f8727d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.844966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ff89d-4b7e-4066-9ac6-f09179f8727d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.845155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ff89d-4b7e-4066-9ac6-f09179f8727d None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.845927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ff89d-4b7e-4066-9ac6-f09179f8727d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:12.846235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ff89d-4b7e-4066-9ac6-f09179f8727d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:12.847717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ff89d-4b7e-4066-9ac6-f09179f8727d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:12.858997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 38/74] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:12.859660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ff89d-4b7e-4066-9ac6-f09179f8727d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:12.900633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278acd5a-2c09-4ae8-88e9-0b8a9de8f3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:12.901674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278acd5a-2c09-4ae8-88e9-0b8a9de8f3e1 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.902145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278acd5a-2c09-4ae8-88e9-0b8a9de8f3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.902595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278acd5a-2c09-4ae8-88e9-0b8a9de8f3e1 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.903928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278acd5a-2c09-4ae8-88e9-0b8a9de8f3e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:12.904395 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278acd5a-2c09-4ae8-88e9-0b8a9de8f3e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:12.906854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278acd5a-2c09-4ae8-88e9-0b8a9de8f3e1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:12.918928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 37/75] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:40:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:12.921166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307101-c4f3-411c-a191-e1df48eab76a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:12.921658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307101-c4f3-411c-a191-e1df48eab76a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:12.922153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f307101-c4f3-411c-a191-e1df48eab76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:12.947233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-5f307101-c4f3-411c-a191-e1df48eab76a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:12.949179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5f307101-c4f3-411c-a191-e1df48eab76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:13.023882 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ff89d-4b7e-4066-9ac6-f09179f8727d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:13.036792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 38/76] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:13.037672 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f307101-c4f3-411c-a191-e1df48eab76a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:13.069763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fce16-329f-4845-a3e9-1f29317e12d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:13.070639 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fce16-329f-4845-a3e9-1f29317e12d8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:13.071166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fce16-329f-4845-a3e9-1f29317e12d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:13.071609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fce16-329f-4845-a3e9-1f29317e12d8 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:13.072634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fce16-329f-4845-a3e9-1f29317e12d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:13.073101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fce16-329f-4845-a3e9-1f29317e12d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:13.078022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fce16-329f-4845-a3e9-1f29317e12d8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:13.087988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 39/77] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:13.089176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fce16-329f-4845-a3e9-1f29317e12d8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:13.205985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f307101-c4f3-411c-a191-e1df48eab76a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:13.251055 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fce16-329f-4845-a3e9-1f29317e12d8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:13.264923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b1659-9000-4d4d-9dfa-3c153843916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:13.266203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b1659-9000-4d4d-9dfa-3c153843916e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:13.266659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b1659-9000-4d4d-9dfa-3c153843916e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:13.267150 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b1659-9000-4d4d-9dfa-3c153843916e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:13.269234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b1659-9000-4d4d-9dfa-3c153843916e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:13.271860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b1659-9000-4d4d-9dfa-3c153843916e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:13.274881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 39/78] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:13.276267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b1659-9000-4d4d-9dfa-3c153843916e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:13.296851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8b348-5ee5-4880-aa64-786b76635111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:13.298160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8b348-5ee5-4880-aa64-786b76635111 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:13.298468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8b348-5ee5-4880-aa64-786b76635111 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:13.298876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8b348-5ee5-4880-aa64-786b76635111 None admin] PATH_INFO: `/v3/users/a0034eb1f585466fae466bc41ea6f933` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:13.300546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa8b348-5ee5-4880-aa64-786b76635111 None admin] RBAC: Authorizing `identity:get_user(user_id=a0034eb1f585466fae466bc41ea6f933)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:13.302621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa8b348-5ee5-4880-aa64-786b76635111 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:13.311214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 40/79] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:13 2022] GET /identity/v3/users/a0034eb1f585466fae466bc41ea6f933 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:13.311214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8b348-5ee5-4880-aa64-786b76635111 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:13.429426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b1659-9000-4d4d-9dfa-3c153843916e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:13.469339 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d497f-2a0e-445e-a95b-d1786e2233a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:13.470309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d497f-2a0e-445e-a95b-d1786e2233a1 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:13.470683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d497f-2a0e-445e-a95b-d1786e2233a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:13.471134 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d497f-2a0e-445e-a95b-d1786e2233a1 None admin] PATH_INFO: `/v3/users/adb0be7f832d47cfa9b23b4532305e94` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:13.486971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d497f-2a0e-445e-a95b-d1786e2233a1 None admin] RBAC: Authorizing `identity:get_user(user_id=adb0be7f832d47cfa9b23b4532305e94)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:13.488495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d497f-2a0e-445e-a95b-d1786e2233a1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:13.488914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8b348-5ee5-4880-aa64-786b76635111 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:13.490269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 40/80] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:13 2022] GET /identity/v3/users/adb0be7f832d47cfa9b23b4532305e94 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:13.492429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d497f-2a0e-445e-a95b-d1786e2233a1 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:13.542391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf2de6-638e-4715-b1f2-aacbe378fbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:13.543817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf2de6-638e-4715-b1f2-aacbe378fbc8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:13.544298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf2de6-638e-4715-b1f2-aacbe378fbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:13.544766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf2de6-638e-4715-b1f2-aacbe378fbc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:13.560227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf2de6-638e-4715-b1f2-aacbe378fbc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-a8bf2de6-638e-4715-b1f2-aacbe378fbc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:13.568332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:13.569535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 41/81] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:13.571048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf2de6-638e-4715-b1f2-aacbe378fbc8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:13.647334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d497f-2a0e-445e-a95b-d1786e2233a1 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:13.693247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88feda-587d-4063-9da2-a92c2b7c3ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:13.694262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88feda-587d-4063-9da2-a92c2b7c3ae6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:13.694614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88feda-587d-4063-9da2-a92c2b7c3ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:13.695001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88feda-587d-4063-9da2-a92c2b7c3ae6 None admin] PATH_INFO: `/v3/users/adb0be7f832d47cfa9b23b4532305e94` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:13.697077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88feda-587d-4063-9da2-a92c2b7c3ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=adb0be7f832d47cfa9b23b4532305e94)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:13.698619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88feda-587d-4063-9da2-a92c2b7c3ae6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:13.700450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 41/82] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:13 2022] GET /identity/v3/users/adb0be7f832d47cfa9b23b4532305e94 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:13.701308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88feda-587d-4063-9da2-a92c2b7c3ae6 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:13.712479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf2de6-638e-4715-b1f2-aacbe378fbc8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:13.767903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642e023-6ff9-43bc-98c1-ed6d0107a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:13.768503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642e023-6ff9-43bc-98c1-ed6d0107a85c None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:13.768792 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642e023-6ff9-43bc-98c1-ed6d0107a85c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:13.769073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642e023-6ff9-43bc-98c1-ed6d0107a85c None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:13.769796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642e023-6ff9-43bc-98c1-ed6d0107a85c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:13.771389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642e023-6ff9-43bc-98c1-ed6d0107a85c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:13.811131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 42/83] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:40:13 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:40:13.814951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642e023-6ff9-43bc-98c1-ed6d0107a85c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:13.849141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88feda-587d-4063-9da2-a92c2b7c3ae6 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:13.901027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6f128-4b35-480d-97af-9b98590647b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:13.902104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6f128-4b35-480d-97af-9b98590647b0 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:13.902568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6f128-4b35-480d-97af-9b98590647b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:13.903028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6f128-4b35-480d-97af-9b98590647b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:13.988650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642e023-6ff9-43bc-98c1-ed6d0107a85c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:14.008265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6f128-4b35-480d-97af-9b98590647b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:14.011107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6f128-4b35-480d-97af-9b98590647b0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:14.032962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154159fb-622c-4c39-8511-585b3eca4624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:14.033526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154159fb-622c-4c39-8511-585b3eca4624 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:14.033807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154159fb-622c-4c39-8511-585b3eca4624 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:14.034093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154159fb-622c-4c39-8511-585b3eca4624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:14.156496 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154159fb-622c-4c39-8511-585b3eca4624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:14.160696 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154159fb-622c-4c39-8511-585b3eca4624 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:14.189873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 42/84] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:14.189873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6f128-4b35-480d-97af-9b98590647b0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:14.335008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 43/85] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:14.336529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154159fb-622c-4c39-8511-585b3eca4624 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:14.354613 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6f128-4b35-480d-97af-9b98590647b0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:14.396463 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b2b2d-160b-43f7-a906-8eea431f6046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:14.397842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b2b2d-160b-43f7-a906-8eea431f6046 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:14.398502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b2b2d-160b-43f7-a906-8eea431f6046 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:14.399234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b2b2d-160b-43f7-a906-8eea431f6046 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:14.400993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b2b2d-160b-43f7-a906-8eea431f6046 None admin] RBAC: Adding query filter params (name=member) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:14.401612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b2b2d-160b-43f7-a906-8eea431f6046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:14.403419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b2b2d-160b-43f7-a906-8eea431f6046 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:14.418405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 43/86] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:14.419762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b2b2d-160b-43f7-a906-8eea431f6046 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:14.513198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154159fb-622c-4c39-8511-585b3eca4624 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:14.545378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b2b2d-160b-43f7-a906-8eea431f6046 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:14.559834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f6bb9-e03d-4185-9e66-c5bc40db86aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:14.560701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6bb9-e03d-4185-9e66-c5bc40db86aa None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:14.561042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6bb9-e03d-4185-9e66-c5bc40db86aa None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:14.561360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6bb9-e03d-4185-9e66-c5bc40db86aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:14.599470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92d014-6a71-407b-a6f3-627d89bb10ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:14.600742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d014-6a71-407b-a6f3-627d89bb10ae None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:14.601230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d014-6a71-407b-a6f3-627d89bb10ae None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:14.601816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d014-6a71-407b-a6f3-627d89bb10ae None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:14.604083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92d014-6a71-407b-a6f3-627d89bb10ae None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:14.606031 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92d014-6a71-407b-a6f3-627d89bb10ae None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:14.608322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 44/87] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:14 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:14.609753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92d014-6a71-407b-a6f3-627d89bb10ae None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:14.656120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f6bb9-e03d-4185-9e66-c5bc40db86aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:14.657728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f6bb9-e03d-4185-9e66-c5bc40db86aa None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:14.741597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92d014-6a71-407b-a6f3-627d89bb10ae None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:14.780006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6834194-18ff-4349-bd90-24d558c96c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:14.781118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6834194-18ff-4349-bd90-24d558c96c38 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:14.781431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6834194-18ff-4349-bd90-24d558c96c38 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:14.781710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6834194-18ff-4349-bd90-24d558c96c38 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:14.783108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6834194-18ff-4349-bd90-24d558c96c38 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:14.784600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6834194-18ff-4349-bd90-24d558c96c38 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:14.786939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 45/88] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:14 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:14.787958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6834194-18ff-4349-bd90-24d558c96c38 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:14.792648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 44/89] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:14.793948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f6bb9-e03d-4185-9e66-c5bc40db86aa None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:14.911055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6834194-18ff-4349-bd90-24d558c96c38 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:14.914521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f6bb9-e03d-4185-9e66-c5bc40db86aa None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:14.953141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b113b-548f-4f8b-98bb-35c769dbbee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:14.954528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b113b-548f-4f8b-98bb-35c769dbbee4 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:14.954766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b113b-548f-4f8b-98bb-35c769dbbee4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:14.955060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b113b-548f-4f8b-98bb-35c769dbbee4 None admin] PATH_INFO: `/v3/users/adb0be7f832d47cfa9b23b4532305e94` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:14.956696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b113b-548f-4f8b-98bb-35c769dbbee4 None admin] RBAC: Authorizing `identity:get_user(user_id=adb0be7f832d47cfa9b23b4532305e94)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:14.958559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b113b-548f-4f8b-98bb-35c769dbbee4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:14.959997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f42bf-4f9b-46a4-83fc-dc66fb5dd4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:14.960755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 46/90] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:14 2022] GET /identity/v3/users/adb0be7f832d47cfa9b23b4532305e94 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:14.961531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f42bf-4f9b-46a4-83fc-dc66fb5dd4b8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:14.962118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f42bf-4f9b-46a4-83fc-dc66fb5dd4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:14.962619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f42bf-4f9b-46a4-83fc-dc66fb5dd4b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:14.963964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f42bf-4f9b-46a4-83fc-dc66fb5dd4b8 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=8ff5716d18b14f8c83813becdf6695bd, user.id=a0034eb1f585466fae466bc41ea6f933) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:14.964568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f42bf-4f9b-46a4-83fc-dc66fb5dd4b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:14.967141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b113b-548f-4f8b-98bb-35c769dbbee4 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:14.967925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f42bf-4f9b-46a4-83fc-dc66fb5dd4b8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:14.974920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 45/91] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:14 2022] GET /identity/v3/role_assignments?user.id=a0034eb1f585466fae466bc41ea6f933&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:14.976143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f42bf-4f9b-46a4-83fc-dc66fb5dd4b8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:15.105367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f42bf-4f9b-46a4-83fc-dc66fb5dd4b8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:15.143620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b113b-548f-4f8b-98bb-35c769dbbee4 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:15.155450 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33ce82-cf78-4b79-a37f-164f361a30a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:15.157047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33ce82-cf78-4b79-a37f-164f361a30a7 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:15.157750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33ce82-cf78-4b79-a37f-164f361a30a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:15.158509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33ce82-cf78-4b79-a37f-164f361a30a7 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:15.160751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33ce82-cf78-4b79-a37f-164f361a30a7 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:15.162817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33ce82-cf78-4b79-a37f-164f361a30a7 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:15.164995 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 46/92] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:14 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:15.166082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33ce82-cf78-4b79-a37f-164f361a30a7 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:15.192166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca63c4e-699b-4d69-90ac-17369bf0b4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:15.193010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca63c4e-699b-4d69-90ac-17369bf0b4a5 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:15.193119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca63c4e-699b-4d69-90ac-17369bf0b4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:15.193399 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca63c4e-699b-4d69-90ac-17369bf0b4a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:15.194806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca63c4e-699b-4d69-90ac-17369bf0b4a5 None admin] RBAC: Adding query filter params (role.id=4501501962414ed0abfba693ba95918c, scope.project.id=8ff5716d18b14f8c83813becdf6695bd, user.id=adb0be7f832d47cfa9b23b4532305e94) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:15.195143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca63c4e-699b-4d69-90ac-17369bf0b4a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:15.197292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca63c4e-699b-4d69-90ac-17369bf0b4a5 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:15.204281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 47/93] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:14 2022] GET /identity/v3/role_assignments?user.id=adb0be7f832d47cfa9b23b4532305e94&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=4501501962414ed0abfba693ba95918c => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:15.205173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca63c4e-699b-4d69-90ac-17369bf0b4a5 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:15.344710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca63c4e-699b-4d69-90ac-17369bf0b4a5 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:15.387876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30dc6f-966d-4ec1-b5ba-7babad746ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:15.388985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30dc6f-966d-4ec1-b5ba-7babad746ebe None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:15.389440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30dc6f-966d-4ec1-b5ba-7babad746ebe None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:15.389739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30dc6f-966d-4ec1-b5ba-7babad746ebe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:15.390600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30dc6f-966d-4ec1-b5ba-7babad746ebe None admin] RBAC: Adding query filter params (role.id=90868071f00340ae853f1188777a2244, scope.project.id=8ff5716d18b14f8c83813becdf6695bd, user.id=adb0be7f832d47cfa9b23b4532305e94) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:15.390975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30dc6f-966d-4ec1-b5ba-7babad746ebe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:15.392511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30dc6f-966d-4ec1-b5ba-7babad746ebe None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:15.400134 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 48/94] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:15 2022] GET /identity/v3/role_assignments?user.id=adb0be7f832d47cfa9b23b4532305e94&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=90868071f00340ae853f1188777a2244 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:15.451848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33ce82-cf78-4b79-a37f-164f361a30a7 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:15.523418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbaa5e-8bc1-4672-8ded-73504d8fa32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:15.524104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbaa5e-8bc1-4672-8ded-73504d8fa32f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:15.524958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbaa5e-8bc1-4672-8ded-73504d8fa32f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:15.525337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbaa5e-8bc1-4672-8ded-73504d8fa32f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:15.631991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbaa5e-8bc1-4672-8ded-73504d8fa32f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:15.633520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbaa5e-8bc1-4672-8ded-73504d8fa32f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:15.761333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 47/95] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:15.767403 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30dc6f-966d-4ec1-b5ba-7babad746ebe None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:15.921571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30dc6f-966d-4ec1-b5ba-7babad746ebe None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:16.022422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e931acf-9d42-47b0-9265-f181b1db69b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:16.024065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931acf-9d42-47b0-9265-f181b1db69b7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:16.024462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931acf-9d42-47b0-9265-f181b1db69b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:16.025709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931acf-9d42-47b0-9265-f181b1db69b7 None admin] PATH_INFO: `/v3/projects/4e1638ccc38b47e39a1be33df7681a22` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:16.032020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e931acf-9d42-47b0-9265-f181b1db69b7 None admin] RBAC: Authorizing `identity:get_project(project_id=4e1638ccc38b47e39a1be33df7681a22)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:16.033968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e931acf-9d42-47b0-9265-f181b1db69b7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:16.037207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 49/96] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:15 2022] GET /identity/v3/projects/4e1638ccc38b47e39a1be33df7681a22 => generated 312 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:16.045530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbaa5e-8bc1-4672-8ded-73504d8fa32f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:16.211164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbaa5e-8bc1-4672-8ded-73504d8fa32f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:16.297083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8751c-409b-4475-b676-92fa00c585a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:16.298678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8751c-409b-4475-b676-92fa00c585a4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:16.299326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8751c-409b-4475-b676-92fa00c585a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:16.299845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8751c-409b-4475-b676-92fa00c585a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:16.301902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8751c-409b-4475-b676-92fa00c585a4 None admin] RBAC: Adding query filter params (role.id=90868071f00340ae853f1188777a2244, scope.project.id=4e1638ccc38b47e39a1be33df7681a22, user.id=adb0be7f832d47cfa9b23b4532305e94) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:16.302477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8751c-409b-4475-b676-92fa00c585a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:16.305464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8751c-409b-4475-b676-92fa00c585a4 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:16.317059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 48/97] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:16 2022] GET /identity/v3/role_assignments?user.id=adb0be7f832d47cfa9b23b4532305e94&scope.project.id=4e1638ccc38b47e39a1be33df7681a22&role.id=90868071f00340ae853f1188777a2244 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:17.434598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c3797-8e56-4088-829b-6e9debe2d3b6 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:17.436137 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c3797-8e56-4088-829b-6e9debe2d3b6 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:17.436902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c3797-8e56-4088-829b-6e9debe2d3b6 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:17.439736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 50/98] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:17.445796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bded7b-916f-40ee-a49a-2e1230b622e4 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:17.447179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bded7b-916f-40ee-a49a-2e1230b622e4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:17.447179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bded7b-916f-40ee-a49a-2e1230b622e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:17.471037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-78bded7b-916f-40ee-a49a-2e1230b622e4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:17.472268 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-78bded7b-916f-40ee-a49a-2e1230b622e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:17.564496 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 49/99] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:17.608977 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c3797-8e56-4088-829b-6e9debe2d3b6 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:17.766803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c3797-8e56-4088-829b-6e9debe2d3b6 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:17.810601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5671a2e-3a9a-4181-a010-6bc30307785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:17.811934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5671a2e-3a9a-4181-a010-6bc30307785e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:17.812487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5671a2e-3a9a-4181-a010-6bc30307785e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:17.812982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5671a2e-3a9a-4181-a010-6bc30307785e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:17.827609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5671a2e-3a9a-4181-a010-6bc30307785e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-f5671a2e-3a9a-4181-a010-6bc30307785e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:17.837748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:17.838671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:17.840009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:17.842924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 51/100] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:17.850369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bded7b-916f-40ee-a49a-2e1230b622e4 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:18.079594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bded7b-916f-40ee-a49a-2e1230b622e4 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:18.099536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48cbd-ca9c-415d-af20-44e7ec95465a None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:18.100049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48cbd-ca9c-415d-af20-44e7ec95465a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:18.100634 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48cbd-ca9c-415d-af20-44e7ec95465a None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:18.103562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 52/101] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:18.110931 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd9486-0f6e-4714-a8be-f180c050756c None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:18.111232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd9486-0f6e-4714-a8be-f180c050756c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:18.111501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd9486-0f6e-4714-a8be-f180c050756c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:18.163289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c6bd9486-0f6e-4714-a8be-f180c050756c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:18.164558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c6bd9486-0f6e-4714-a8be-f180c050756c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:18.189441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97c224-800d-4252-b218-9d78d712fea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:18.190146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97c224-800d-4252-b218-9d78d712fea9 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:18.190423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97c224-800d-4252-b218-9d78d712fea9 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:18.190732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97c224-800d-4252-b218-9d78d712fea9 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:18.191497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b97c224-800d-4252-b218-9d78d712fea9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:18.191807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b97c224-800d-4252-b218-9d78d712fea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:18.193233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b97c224-800d-4252-b218-9d78d712fea9 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:18.212828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 50/102] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:18.222606 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97c224-800d-4252-b218-9d78d712fea9 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:18.387155 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 53/103] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:18.467450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd9486-0f6e-4714-a8be-f180c050756c None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:18.488115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97c224-800d-4252-b218-9d78d712fea9 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:18.609571 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d020ba-7d21-447a-8217-0faeb7e1c688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:18.611165 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d020ba-7d21-447a-8217-0faeb7e1c688 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:18.611642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d020ba-7d21-447a-8217-0faeb7e1c688 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:18.612087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d020ba-7d21-447a-8217-0faeb7e1c688 None admin] PATH_INFO: `/v3/users/5cf1ec9c433941228e63ec46cbc595eb` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:18.653839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d020ba-7d21-447a-8217-0faeb7e1c688 None admin] RBAC: Authorizing `identity:get_user(user_id=5cf1ec9c433941228e63ec46cbc595eb)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:18.655445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d020ba-7d21-447a-8217-0faeb7e1c688 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:18.657563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 51/104] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:18 2022] GET /identity/v3/users/5cf1ec9c433941228e63ec46cbc595eb => generated 289 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:18.663903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d020ba-7d21-447a-8217-0faeb7e1c688 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:18.694883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd9486-0f6e-4714-a8be-f180c050756c None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:18.771420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874a8a1-34a4-4f8b-9864-412db70bd406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:18.772875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874a8a1-34a4-4f8b-9864-412db70bd406 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:18.773392 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874a8a1-34a4-4f8b-9864-412db70bd406 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:18.773905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874a8a1-34a4-4f8b-9864-412db70bd406 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:18.776859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874a8a1-34a4-4f8b-9864-412db70bd406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:18.782750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874a8a1-34a4-4f8b-9864-412db70bd406 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:18.790734 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 54/105] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:18.814823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874a8a1-34a4-4f8b-9864-412db70bd406 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:18.950972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d020ba-7d21-447a-8217-0faeb7e1c688 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:19.004734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9347147-9644-4d2a-875e-1afd1ae0a4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:19.005447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9347147-9644-4d2a-875e-1afd1ae0a4a8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:19.005736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9347147-9644-4d2a-875e-1afd1ae0a4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:19.006007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9347147-9644-4d2a-875e-1afd1ae0a4a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:19.101777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9347147-9644-4d2a-875e-1afd1ae0a4a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:19.104294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9347147-9644-4d2a-875e-1afd1ae0a4a8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:19.161704 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874a8a1-34a4-4f8b-9864-412db70bd406 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:19.263669 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48596430-2a78-4594-8563-fe43d80cc080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:19.264750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48596430-2a78-4594-8563-fe43d80cc080 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:19.265246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48596430-2a78-4594-8563-fe43d80cc080 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:19.265739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48596430-2a78-4594-8563-fe43d80cc080 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:19.266680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48596430-2a78-4594-8563-fe43d80cc080 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:19.268421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48596430-2a78-4594-8563-fe43d80cc080 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:19.271374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 52/106] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:19.279559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9347147-9644-4d2a-875e-1afd1ae0a4a8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:19.323345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 55/107] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:40:18 2022] POST /identity/v3/projects => generated 302 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:40:19.503754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9347147-9644-4d2a-875e-1afd1ae0a4a8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:19.558345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e29d1-3a37-4a1d-b5fb-e10b9591f9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:19.559209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e29d1-3a37-4a1d-b5fb-e10b9591f9ba None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:19.559516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e29d1-3a37-4a1d-b5fb-e10b9591f9ba None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:19.559797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e29d1-3a37-4a1d-b5fb-e10b9591f9ba None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:19.562857 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e29d1-3a37-4a1d-b5fb-e10b9591f9ba None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:19.564517 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e29d1-3a37-4a1d-b5fb-e10b9591f9ba None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:19.566617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 53/108] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:19 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:19.574513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48596430-2a78-4594-8563-fe43d80cc080 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:19.831375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48596430-2a78-4594-8563-fe43d80cc080 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:19.893075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae433d2-fce9-47d0-bb6b-5d65cc03f3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:19.894275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae433d2-fce9-47d0-bb6b-5d65cc03f3fd None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:19.894813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae433d2-fce9-47d0-bb6b-5d65cc03f3fd None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:19.895412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae433d2-fce9-47d0-bb6b-5d65cc03f3fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:19.897251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae433d2-fce9-47d0-bb6b-5d65cc03f3fd None admin] RBAC: Adding query filter params (role.id=a00af3df784a4cbf9d4a59157edebd16, scope.project.id=8ff5716d18b14f8c83813becdf6695bd, user.id=5cf1ec9c433941228e63ec46cbc595eb) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:19.897787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae433d2-fce9-47d0-bb6b-5d65cc03f3fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:19.901223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae433d2-fce9-47d0-bb6b-5d65cc03f3fd None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:19.912938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 56/109] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:19 2022] GET /identity/v3/role_assignments?user.id=5cf1ec9c433941228e63ec46cbc595eb&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=a00af3df784a4cbf9d4a59157edebd16 => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:20.065360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70518f6c-32e1-4285-86be-fda223377da0 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:20.065991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70518f6c-32e1-4285-86be-fda223377da0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:20.066530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70518f6c-32e1-4285-86be-fda223377da0 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:20.069343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 54/110] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:20.085818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f103390-4622-49eb-8a42-9b4300ebd57b None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:20.086239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f103390-4622-49eb-8a42-9b4300ebd57b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:20.087424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f103390-4622-49eb-8a42-9b4300ebd57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:20.121831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-8f103390-4622-49eb-8a42-9b4300ebd57b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:20.123192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8f103390-4622-49eb-8a42-9b4300ebd57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:20.139273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe171286-efaa-44d2-a586-b4e834f5dceb None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:20.140462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe171286-efaa-44d2-a586-b4e834f5dceb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:20.141025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe171286-efaa-44d2-a586-b4e834f5dceb None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:20.143145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 55/111] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:20.160180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fbda5-1fa2-4039-9fff-ef0130aa1484 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:20.161052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fbda5-1fa2-4039-9fff-ef0130aa1484 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:20.161761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fbda5-1fa2-4039-9fff-ef0130aa1484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:20.204456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-c55fbda5-1fa2-4039-9fff-ef0130aa1484 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:20.206886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c55fbda5-1fa2-4039-9fff-ef0130aa1484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:20.302271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 57/112] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:20.347733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f103390-4622-49eb-8a42-9b4300ebd57b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:20.356966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 56/113] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:20.418083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fbda5-1fa2-4039-9fff-ef0130aa1484 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:20.492918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f103390-4622-49eb-8a42-9b4300ebd57b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:20.546555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad43dbf-351f-4c57-b8f6-188a4b38bdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:20.548081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad43dbf-351f-4c57-b8f6-188a4b38bdef None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:20.548824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad43dbf-351f-4c57-b8f6-188a4b38bdef None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:20.549428 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad43dbf-351f-4c57-b8f6-188a4b38bdef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:20.591823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad43dbf-351f-4c57-b8f6-188a4b38bdef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:20.610156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fbda5-1fa2-4039-9fff-ef0130aa1484 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-4ad43dbf-351f-4c57-b8f6-188a4b38bdef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:20.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:20.614519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:20.615467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:20.624867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 58/114] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:40:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:40:20.629358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adda03-d47a-4542-8072-0e762ef9248e None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:20.629863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adda03-d47a-4542-8072-0e762ef9248e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:20.630375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adda03-d47a-4542-8072-0e762ef9248e None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:20.633590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 59/115] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:20.643387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4adda03-d47a-4542-8072-0e762ef9248e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:20.654295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30972d7a-383a-49e7-812b-6d69a4047bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:20.655757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30972d7a-383a-49e7-812b-6d69a4047bef None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:20.656266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30972d7a-383a-49e7-812b-6d69a4047bef None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:20.656771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30972d7a-383a-49e7-812b-6d69a4047bef None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:20.689561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30972d7a-383a-49e7-812b-6d69a4047bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-30972d7a-383a-49e7-812b-6d69a4047bef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:20.698005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:20.698987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:20.699738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 57/116] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:40:20.705787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597949d-d332-43bd-bea9-8a02e37b3035 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:20.706072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597949d-d332-43bd-bea9-8a02e37b3035 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:20.706345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597949d-d332-43bd-bea9-8a02e37b3035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:20.724985 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-3597949d-d332-43bd-bea9-8a02e37b3035 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:20.726355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-3597949d-d332-43bd-bea9-8a02e37b3035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:20.830360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 58/117] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:20.831922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597949d-d332-43bd-bea9-8a02e37b3035 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:20.879598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4adda03-d47a-4542-8072-0e762ef9248e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:20.955280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e55a0-5c60-456d-8e91-9b1aa8340a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:20.956379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e55a0-5c60-456d-8e91-9b1aa8340a92 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:20.956875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e55a0-5c60-456d-8e91-9b1aa8340a92 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:20.957359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e55a0-5c60-456d-8e91-9b1aa8340a92 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:20.958747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e55a0-5c60-456d-8e91-9b1aa8340a92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:20.959290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e55a0-5c60-456d-8e91-9b1aa8340a92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:20.961009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e55a0-5c60-456d-8e91-9b1aa8340a92 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:20.993684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 60/118] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:20.995475 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e55a0-5c60-456d-8e91-9b1aa8340a92 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:21.147441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e55a0-5c60-456d-8e91-9b1aa8340a92 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:21.177983 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597949d-d332-43bd-bea9-8a02e37b3035 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:21.209560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3c902-942b-4bab-bfe1-16eb9396a42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:21.210793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3c902-942b-4bab-bfe1-16eb9396a42c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:21.211212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3c902-942b-4bab-bfe1-16eb9396a42c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:21.211678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3c902-942b-4bab-bfe1-16eb9396a42c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:21.242201 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3c902-942b-4bab-bfe1-16eb9396a42c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-e9f3c902-942b-4bab-bfe1-16eb9396a42c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:21.254487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:21.255456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:21.256202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:21.256782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 61/119] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:40:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:21.257993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3c902-942b-4bab-bfe1-16eb9396a42c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:21.275935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c4d7b-7a72-4b16-b83e-811f88e8cf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:21.277054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c4d7b-7a72-4b16-b83e-811f88e8cf0a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:21.277533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c4d7b-7a72-4b16-b83e-811f88e8cf0a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:21.277987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c4d7b-7a72-4b16-b83e-811f88e8cf0a None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:21.279336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c4d7b-7a72-4b16-b83e-811f88e8cf0a None admin] RBAC: Adding query filter params (name=member) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:21.279845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c4d7b-7a72-4b16-b83e-811f88e8cf0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:21.282669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c4d7b-7a72-4b16-b83e-811f88e8cf0a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:21.302318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 59/120] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:21.304522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7d5b6-b0d2-4356-9c5c-b7ebdea65ac2 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:21.304987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7d5b6-b0d2-4356-9c5c-b7ebdea65ac2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:21.305481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7d5b6-b0d2-4356-9c5c-b7ebdea65ac2 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:21.313698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 60/121] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:21.315346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7d5b6-b0d2-4356-9c5c-b7ebdea65ac2 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:21.449911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3c902-942b-4bab-bfe1-16eb9396a42c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:21.509598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7d5b6-b0d2-4356-9c5c-b7ebdea65ac2 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:21.515115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735d607-215e-479f-a43a-254c7ad3084e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:21.516490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735d607-215e-479f-a43a-254c7ad3084e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:21.517026 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735d607-215e-479f-a43a-254c7ad3084e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:21.517525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735d607-215e-479f-a43a-254c7ad3084e None admin] PATH_INFO: `/v3/users/a0034eb1f585466fae466bc41ea6f933` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:21.519319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735d607-215e-479f-a43a-254c7ad3084e None admin] RBAC: Authorizing `identity:get_user(user_id=a0034eb1f585466fae466bc41ea6f933)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:21.521591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735d607-215e-479f-a43a-254c7ad3084e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:21.523885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 62/122] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:21 2022] GET /identity/v3/users/a0034eb1f585466fae466bc41ea6f933 => generated 247 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:21.530462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735d607-215e-479f-a43a-254c7ad3084e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:21.579215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0463eb-3ec7-4a28-8db1-5ef859220db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:21.580438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0463eb-3ec7-4a28-8db1-5ef859220db7 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:21.580910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0463eb-3ec7-4a28-8db1-5ef859220db7 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:21.581423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0463eb-3ec7-4a28-8db1-5ef859220db7 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:21.582920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0463eb-3ec7-4a28-8db1-5ef859220db7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:21.583297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0463eb-3ec7-4a28-8db1-5ef859220db7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:21.585924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0463eb-3ec7-4a28-8db1-5ef859220db7 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:21.600553 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 61/123] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:40:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:21.610386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080b3bb-284f-4934-85c1-9d3b6ffc1ca4 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:21.611412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080b3bb-284f-4934-85c1-9d3b6ffc1ca4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:21.611581 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080b3bb-284f-4934-85c1-9d3b6ffc1ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:21.639639 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f080b3bb-284f-4934-85c1-9d3b6ffc1ca4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:21.641903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f080b3bb-284f-4934-85c1-9d3b6ffc1ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:21.693242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735d607-215e-479f-a43a-254c7ad3084e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:21.733181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98358543-e564-43f2-b420-e44cdc7465ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:21.734413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98358543-e564-43f2-b420-e44cdc7465ab None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:21.734939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98358543-e564-43f2-b420-e44cdc7465ab None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:21.735406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98358543-e564-43f2-b420-e44cdc7465ab None admin] PATH_INFO: `/v3/users/adb0be7f832d47cfa9b23b4532305e94` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:21.736906 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98358543-e564-43f2-b420-e44cdc7465ab None admin] RBAC: Authorizing `identity:get_user(user_id=adb0be7f832d47cfa9b23b4532305e94)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:21.738580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98358543-e564-43f2-b420-e44cdc7465ab None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:21.740511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 63/124] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:21 2022] GET /identity/v3/users/adb0be7f832d47cfa9b23b4532305e94 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:21.741725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98358543-e564-43f2-b420-e44cdc7465ab None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:21.768662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 62/125] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:21.770298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080b3bb-284f-4934-85c1-9d3b6ffc1ca4 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:21.857990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98358543-e564-43f2-b420-e44cdc7465ab None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:21.897421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312fa51-9980-4d33-85e6-66406174e043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:21.898746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312fa51-9980-4d33-85e6-66406174e043 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:21.899858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312fa51-9980-4d33-85e6-66406174e043 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:21.900707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312fa51-9980-4d33-85e6-66406174e043 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:21.902371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312fa51-9980-4d33-85e6-66406174e043 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:21.904169 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312fa51-9980-4d33-85e6-66406174e043 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:21.906355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 64/126] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:21 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:21.912227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312fa51-9980-4d33-85e6-66406174e043 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:21.997916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080b3bb-284f-4934-85c1-9d3b6ffc1ca4 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:22.032542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312fa51-9980-4d33-85e6-66406174e043 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:22.062610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d725-1560-4702-a68e-5ef2d2012c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:22.064090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d725-1560-4702-a68e-5ef2d2012c80 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:22.064564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d725-1560-4702-a68e-5ef2d2012c80 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:22.065030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d725-1560-4702-a68e-5ef2d2012c80 None admin] PATH_INFO: `/v3/users/adb0be7f832d47cfa9b23b4532305e94` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:22.067822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946d725-1560-4702-a68e-5ef2d2012c80 None admin] RBAC: Authorizing `identity:get_user(user_id=adb0be7f832d47cfa9b23b4532305e94)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:22.070181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946d725-1560-4702-a68e-5ef2d2012c80 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:22.073746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 63/127] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:21 2022] GET /identity/v3/users/adb0be7f832d47cfa9b23b4532305e94 => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:22.074747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d725-1560-4702-a68e-5ef2d2012c80 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:22.086340 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210335e1-71b0-43af-8c87-a2520fdc3e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:22.088049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210335e1-71b0-43af-8c87-a2520fdc3e9b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:22.088627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210335e1-71b0-43af-8c87-a2520fdc3e9b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:22.089103 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210335e1-71b0-43af-8c87-a2520fdc3e9b None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:22.091938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210335e1-71b0-43af-8c87-a2520fdc3e9b None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:22.095161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210335e1-71b0-43af-8c87-a2520fdc3e9b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:22.098843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 65/128] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:21 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:22.101482 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210335e1-71b0-43af-8c87-a2520fdc3e9b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:22.244737 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d725-1560-4702-a68e-5ef2d2012c80 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:22.256166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210335e1-71b0-43af-8c87-a2520fdc3e9b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:22.293904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac2c1f-7190-4888-a161-33c0ee09a786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:22.294784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2c1f-7190-4888-a161-33c0ee09a786 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:22.295247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2c1f-7190-4888-a161-33c0ee09a786 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:22.295538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac2c1f-7190-4888-a161-33c0ee09a786 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:22.309893 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efab407-5a7f-451c-9fed-1f14409ae5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:22.312253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efab407-5a7f-451c-9fed-1f14409ae5d0 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:22.313176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efab407-5a7f-451c-9fed-1f14409ae5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:22.314335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efab407-5a7f-451c-9fed-1f14409ae5d0 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd/users/a0034eb1f585466fae466bc41ea6f933/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:22.314846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac2c1f-7190-4888-a161-33c0ee09a786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:22.318954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efab407-5a7f-451c-9fed-1f14409ae5d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5716d18b14f8c83813becdf6695bd, user_id=a0034eb1f585466fae466bc41ea6f933, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:22.322332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efab407-5a7f-451c-9fed-1f14409ae5d0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-2bac2c1f-7190-4888-a161-33c0ee09a786 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:22.328532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:22.330280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:22.331704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 64/129] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:22.333070 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac2c1f-7190-4888-a161-33c0ee09a786 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:22.360890 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 66/130] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:40:22 2022] PUT /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd/users/a0034eb1f585466fae466bc41ea6f933/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:40:22.363142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efab407-5a7f-451c-9fed-1f14409ae5d0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:22.530856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac2c1f-7190-4888-a161-33c0ee09a786 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:22.580026 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efab407-5a7f-451c-9fed-1f14409ae5d0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:22.598803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc767a6-0f05-40db-9298-e2091f809e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:22.600411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc767a6-0f05-40db-9298-e2091f809e4f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:22.600885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc767a6-0f05-40db-9298-e2091f809e4f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:22.601325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc767a6-0f05-40db-9298-e2091f809e4f None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:22.603828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc767a6-0f05-40db-9298-e2091f809e4f None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:22.606758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc767a6-0f05-40db-9298-e2091f809e4f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:22.615012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 65/131] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:22 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:22.616862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc767a6-0f05-40db-9298-e2091f809e4f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:22.651060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1df238-1523-4c5d-b9aa-535b0ec88eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:22.652709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1df238-1523-4c5d-b9aa-535b0ec88eca None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:22.653220 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1df238-1523-4c5d-b9aa-535b0ec88eca None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:22.653734 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1df238-1523-4c5d-b9aa-535b0ec88eca None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd/users/adb0be7f832d47cfa9b23b4532305e94/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:22.657925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1df238-1523-4c5d-b9aa-535b0ec88eca None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5716d18b14f8c83813becdf6695bd, user_id=adb0be7f832d47cfa9b23b4532305e94, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:22.660559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1df238-1523-4c5d-b9aa-535b0ec88eca None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:22.691986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 67/132] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:40:22 2022] PUT /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd/users/adb0be7f832d47cfa9b23b4532305e94/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:40:22.694136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1df238-1523-4c5d-b9aa-535b0ec88eca None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:22.810883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc767a6-0f05-40db-9298-e2091f809e4f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:22.858015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe84c1d-451c-4280-8f1a-fa81363704a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:22.858746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84c1d-451c-4280-8f1a-fa81363704a3 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:22.859043 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84c1d-451c-4280-8f1a-fa81363704a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:22.859321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84c1d-451c-4280-8f1a-fa81363704a3 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:22.860112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe84c1d-451c-4280-8f1a-fa81363704a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:22.860407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe84c1d-451c-4280-8f1a-fa81363704a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:22.862096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe84c1d-451c-4280-8f1a-fa81363704a3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:22.872681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 66/133] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:22.878837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe84c1d-451c-4280-8f1a-fa81363704a3 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:22.906509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1df238-1523-4c5d-b9aa-535b0ec88eca None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:22.976143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d4f50-e7bb-4223-86c1-5fb3d3617fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:22.979222 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d4f50-e7bb-4223-86c1-5fb3d3617fd7 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:22.979222 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d4f50-e7bb-4223-86c1-5fb3d3617fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:22.979222 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d4f50-e7bb-4223-86c1-5fb3d3617fd7 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd/users/adb0be7f832d47cfa9b23b4532305e94/roles/4501501962414ed0abfba693ba95918c` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:22.987762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d4f50-e7bb-4223-86c1-5fb3d3617fd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5716d18b14f8c83813becdf6695bd, user_id=adb0be7f832d47cfa9b23b4532305e94, role_id=4501501962414ed0abfba693ba95918c)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:22.987762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d4f50-e7bb-4223-86c1-5fb3d3617fd7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:23.007236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 68/134] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:40:22 2022] PUT /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd/users/adb0be7f832d47cfa9b23b4532305e94/roles/4501501962414ed0abfba693ba95918c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:40:23.058173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe84c1d-451c-4280-8f1a-fa81363704a3 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:23.165906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57261a4-10f7-443a-8593-3d204919b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:23.166734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57261a4-10f7-443a-8593-3d204919b3de None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:23.167041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57261a4-10f7-443a-8593-3d204919b3de None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:23.167320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57261a4-10f7-443a-8593-3d204919b3de None admin] PATH_INFO: `/v3/users/adb0be7f832d47cfa9b23b4532305e94` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:23.171754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57261a4-10f7-443a-8593-3d204919b3de None admin] RBAC: Authorizing `identity:get_user(user_id=adb0be7f832d47cfa9b23b4532305e94)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:23.173288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57261a4-10f7-443a-8593-3d204919b3de None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:23.175501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 67/135] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:22 2022] GET /identity/v3/users/adb0be7f832d47cfa9b23b4532305e94 => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:23.184528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d4f50-e7bb-4223-86c1-5fb3d3617fd7 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:23.366406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d4f50-e7bb-4223-86c1-5fb3d3617fd7 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:23.439624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beae65-4eda-42b0-a26d-93d97f08ddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:23.440938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beae65-4eda-42b0-a26d-93d97f08ddde None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:23.441475 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beae65-4eda-42b0-a26d-93d97f08ddde None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:23.441963 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beae65-4eda-42b0-a26d-93d97f08ddde None admin] PATH_INFO: `/v3/projects/4e1638ccc38b47e39a1be33df7681a22` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:23.444229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beae65-4eda-42b0-a26d-93d97f08ddde None admin] RBAC: Authorizing `identity:get_project(project_id=4e1638ccc38b47e39a1be33df7681a22)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:23.446782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beae65-4eda-42b0-a26d-93d97f08ddde None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:23.448365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa4ee9-ad2d-4bdd-a24c-de9934d52609 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:23.448657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa4ee9-ad2d-4bdd-a24c-de9934d52609 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:23.449061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa4ee9-ad2d-4bdd-a24c-de9934d52609 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:23.450303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 69/136] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:23 2022] GET /identity/v3/projects/4e1638ccc38b47e39a1be33df7681a22 => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:23.450420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 68/137] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:23.458054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325bde-d711-425c-8c43-0f6b9b9a0328 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:23.458407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325bde-d711-425c-8c43-0f6b9b9a0328 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:23.458839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325bde-d711-425c-8c43-0f6b9b9a0328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:23.459716 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beae65-4eda-42b0-a26d-93d97f08ddde None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:23.508127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0c325bde-d711-425c-8c43-0f6b9b9a0328 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:23.509875 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0c325bde-d711-425c-8c43-0f6b9b9a0328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:23.665770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beae65-4eda-42b0-a26d-93d97f08ddde None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:23.708153 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 69/138] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:23.755710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2bb1c-4910-4334-92e0-4f10f76e3110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:23.757373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2bb1c-4910-4334-92e0-4f10f76e3110 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:23.757888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2bb1c-4910-4334-92e0-4f10f76e3110 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:23.758433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2bb1c-4910-4334-92e0-4f10f76e3110 None admin] PATH_INFO: `/v3/projects/4e1638ccc38b47e39a1be33df7681a22/users/adb0be7f832d47cfa9b23b4532305e94/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:23.769099 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2bb1c-4910-4334-92e0-4f10f76e3110 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e1638ccc38b47e39a1be33df7681a22, user_id=adb0be7f832d47cfa9b23b4532305e94, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:23.771972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2bb1c-4910-4334-92e0-4f10f76e3110 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:23.802635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 70/139] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:40:23 2022] PUT /identity/v3/projects/4e1638ccc38b47e39a1be33df7681a22/users/adb0be7f832d47cfa9b23b4532305e94/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:40:23.815691 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c325bde-d711-425c-8c43-0f6b9b9a0328 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:23.965637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20420d78-3104-4753-93f5-e557032cfd16 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:23.966037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20420d78-3104-4753-93f5-e557032cfd16 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:23.966529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20420d78-3104-4753-93f5-e557032cfd16 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:23.969277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 71/140] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:23.979271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c4c5-6ae0-477a-a761-b1422f34249a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:23.979740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c4c5-6ae0-477a-a761-b1422f34249a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:23.979978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c325bde-d711-425c-8c43-0f6b9b9a0328 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:23.980151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4c4c5-6ae0-477a-a761-b1422f34249a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:24.028590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f30c09-a33f-4434-9c73-b50ef602e85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:24.029744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f30c09-a33f-4434-9c73-b50ef602e85d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:24.030018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f30c09-a33f-4434-9c73-b50ef602e85d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:24.030293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f30c09-a33f-4434-9c73-b50ef602e85d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:24.032187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f30c09-a33f-4434-9c73-b50ef602e85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:24.033976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f30c09-a33f-4434-9c73-b50ef602e85d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:24.036437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 70/141] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:24.042550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-6fd4c4c5-6ae0-477a-a761-b1422f34249a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:24.043607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f30c09-a33f-4434-9c73-b50ef602e85d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:24.044795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6fd4c4c5-6ae0-477a-a761-b1422f34249a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:24.238580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f30c09-a33f-4434-9c73-b50ef602e85d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:24.256609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 72/142] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:24.304744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4c4c5-6ae0-477a-a761-b1422f34249a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:24.305961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff7ef8-f483-463b-a5c1-a4060501da4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:24.307152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7ef8-f483-463b-a5c1-a4060501da4c None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:24.307650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7ef8-f483-463b-a5c1-a4060501da4c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:24.308132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7ef8-f483-463b-a5c1-a4060501da4c None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:24.309487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff7ef8-f483-463b-a5c1-a4060501da4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:24.312326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff7ef8-f483-463b-a5c1-a4060501da4c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:24.359602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 71/143] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:40:24 2022] POST /identity/v3/users => generated 283 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:40:24.504850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4c4c5-6ae0-477a-a761-b1422f34249a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:24.553357 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476a063-c477-4eb6-9306-99e793f6c02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:24.554204 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476a063-c477-4eb6-9306-99e793f6c02c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:24.554604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476a063-c477-4eb6-9306-99e793f6c02c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:24.554953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476a063-c477-4eb6-9306-99e793f6c02c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:24.570335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476a063-c477-4eb6-9306-99e793f6c02c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-c476a063-c477-4eb6-9306-99e793f6c02c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:24.579697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:24.582051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:24.585355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 73/144] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:40:24.596240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff7ef8-f483-463b-a5c1-a4060501da4c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:24.778981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff7ef8-f483-463b-a5c1-a4060501da4c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:24.845353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc059f3-37e2-4dc5-885c-38d84ad8c60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:24.846615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc059f3-37e2-4dc5-885c-38d84ad8c60a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:24.851459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc059f3-37e2-4dc5-885c-38d84ad8c60a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:24.852026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc059f3-37e2-4dc5-885c-38d84ad8c60a None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:24.853807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc059f3-37e2-4dc5-885c-38d84ad8c60a None admin] RBAC: Adding query filter params (name=reader) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:24.854347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc059f3-37e2-4dc5-885c-38d84ad8c60a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:24.857293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc059f3-37e2-4dc5-885c-38d84ad8c60a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:24.879339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 72/145] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:24.885754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476a063-c477-4eb6-9306-99e793f6c02c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:25.070149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476a063-c477-4eb6-9306-99e793f6c02c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:25.136869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48cba2-db17-444a-b5dc-d3df04f0977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:25.138374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48cba2-db17-444a-b5dc-d3df04f0977f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:25.138866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48cba2-db17-444a-b5dc-d3df04f0977f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:25.139315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48cba2-db17-444a-b5dc-d3df04f0977f None admin] PATH_INFO: `/v3/users/5cf1ec9c433941228e63ec46cbc595eb` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:25.144160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48cba2-db17-444a-b5dc-d3df04f0977f None admin] RBAC: Authorizing `identity:get_user(user_id=5cf1ec9c433941228e63ec46cbc595eb)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:25.146655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48cba2-db17-444a-b5dc-d3df04f0977f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:25.150575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 74/146] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:24 2022] GET /identity/v3/users/5cf1ec9c433941228e63ec46cbc595eb => generated 289 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:25.159923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc059f3-37e2-4dc5-885c-38d84ad8c60a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:25.345412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc059f3-37e2-4dc5-885c-38d84ad8c60a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:25.401712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331bc9c-2da9-4ff1-80b4-744b7f61f480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:25.402673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331bc9c-2da9-4ff1-80b4-744b7f61f480 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:25.403025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331bc9c-2da9-4ff1-80b4-744b7f61f480 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:25.403306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331bc9c-2da9-4ff1-80b4-744b7f61f480 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:25.407861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331bc9c-2da9-4ff1-80b4-744b7f61f480 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:25.410061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331bc9c-2da9-4ff1-80b4-744b7f61f480 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:25.411967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 73/147] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:25 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:25.419352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48cba2-db17-444a-b5dc-d3df04f0977f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:25.635687 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48cba2-db17-444a-b5dc-d3df04f0977f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:25.697502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004fe65-c83e-4087-a03f-95cd4464a3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:25.699030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004fe65-c83e-4087-a03f-95cd4464a3b1 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:25.699495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004fe65-c83e-4087-a03f-95cd4464a3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:25.699945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004fe65-c83e-4087-a03f-95cd4464a3b1 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd/users/5cf1ec9c433941228e63ec46cbc595eb/roles/a00af3df784a4cbf9d4a59157edebd16` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:25.709199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004fe65-c83e-4087-a03f-95cd4464a3b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5716d18b14f8c83813becdf6695bd, user_id=5cf1ec9c433941228e63ec46cbc595eb, role_id=a00af3df784a4cbf9d4a59157edebd16)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:25.712545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004fe65-c83e-4087-a03f-95cd4464a3b1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:25.738755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 75/148] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:40:25 2022] PUT /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd/users/5cf1ec9c433941228e63ec46cbc595eb/roles/a00af3df784a4cbf9d4a59157edebd16 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:40:26.838977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c8b11-7445-409e-970a-9db8ef5bd4eb None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:26.839764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c8b11-7445-409e-970a-9db8ef5bd4eb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:26.839931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c8b11-7445-409e-970a-9db8ef5bd4eb None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:26.841903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 74/149] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:26.850435 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4b3fc-a21f-4285-83a3-f469e29d71da None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:26.851385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4b3fc-a21f-4285-83a3-f469e29d71da None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:26.851385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4b3fc-a21f-4285-83a3-f469e29d71da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:26.883945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-41f4b3fc-a21f-4285-83a3-f469e29d71da None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:26.885191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-41f4b3fc-a21f-4285-83a3-f469e29d71da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:26.968258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 76/150] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 13 13:40:27.009696 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c8b11-7445-409e-970a-9db8ef5bd4eb None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:27.138069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c8b11-7445-409e-970a-9db8ef5bd4eb None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:27.188803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863706a-a409-420d-8d0b-d5fc37030269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:27.189649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863706a-a409-420d-8d0b-d5fc37030269 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:27.189937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863706a-a409-420d-8d0b-d5fc37030269 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:27.190215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863706a-a409-420d-8d0b-d5fc37030269 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:27.209554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863706a-a409-420d-8d0b-d5fc37030269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-9863706a-a409-420d-8d0b-d5fc37030269 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:27.218128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:27.219097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:27.219858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:27.223633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 75/151] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:40:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:27.230014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4b3fc-a21f-4285-83a3-f469e29d71da None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:27.360286 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4b3fc-a21f-4285-83a3-f469e29d71da None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:27.413232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb45bc-59f6-4d3c-9f33-a39faf78896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:27.413817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb45bc-59f6-4d3c-9f33-a39faf78896e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:27.414101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb45bc-59f6-4d3c-9f33-a39faf78896e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:27.414377 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb45bc-59f6-4d3c-9f33-a39faf78896e None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:27.415948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb45bc-59f6-4d3c-9f33-a39faf78896e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:27.416259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb45bc-59f6-4d3c-9f33-a39faf78896e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:27.417667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb45bc-59f6-4d3c-9f33-a39faf78896e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:27.431867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 77/152] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:27.437754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863706a-a409-420d-8d0b-d5fc37030269 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:27.569336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863706a-a409-420d-8d0b-d5fc37030269 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:27.614759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d633d-4c92-479b-a166-4779aaf4de74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:27.616077 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d633d-4c92-479b-a166-4779aaf4de74 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:27.616356 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d633d-4c92-479b-a166-4779aaf4de74 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:27.616636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d633d-4c92-479b-a166-4779aaf4de74 None admin] PATH_INFO: `/v3/users/a0034eb1f585466fae466bc41ea6f933` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:27.618141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d633d-4c92-479b-a166-4779aaf4de74 None admin] RBAC: Authorizing `identity:get_user(user_id=a0034eb1f585466fae466bc41ea6f933)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:27.619692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d633d-4c92-479b-a166-4779aaf4de74 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:27.621444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 76/153] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:27 2022] GET /identity/v3/users/a0034eb1f585466fae466bc41ea6f933 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:27.630645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb45bc-59f6-4d3c-9f33-a39faf78896e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:27.752864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb45bc-59f6-4d3c-9f33-a39faf78896e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:27.796291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cab8a-da63-4e20-b538-589afd05e2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:27.796975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cab8a-da63-4e20-b538-589afd05e2da None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:27.797239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cab8a-da63-4e20-b538-589afd05e2da None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:27.797524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cab8a-da63-4e20-b538-589afd05e2da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:27.829500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f4f16-502f-44e5-ad4f-1ab2fa9ab342 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:27.829923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f4f16-502f-44e5-ad4f-1ab2fa9ab342 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:27.830187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f4f16-502f-44e5-ad4f-1ab2fa9ab342 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:27.831579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 77/154] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:40:27.838663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572dcea-482c-49e5-82ed-63571f5938bc None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:27.839196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572dcea-482c-49e5-82ed-63571f5938bc None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:27.840178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572dcea-482c-49e5-82ed-63571f5938bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:27.867874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6572dcea-482c-49e5-82ed-63571f5938bc None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:27.869142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6572dcea-482c-49e5-82ed-63571f5938bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:27.948023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cab8a-da63-4e20-b538-589afd05e2da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:27.949883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cab8a-da63-4e20-b538-589afd05e2da None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:28.077309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 78/155] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:28.125078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572dcea-482c-49e5-82ed-63571f5938bc None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:28.172481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 78/156] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:40:28.178399 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cab8a-da63-4e20-b538-589afd05e2da None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:28.417782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cab8a-da63-4e20-b538-589afd05e2da None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:28.429283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572dcea-482c-49e5-82ed-63571f5938bc None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:28.515050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9a623-e79e-495c-98fd-b5ff20588f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:28.516508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9a623-e79e-495c-98fd-b5ff20588f08 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:28.517677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9a623-e79e-495c-98fd-b5ff20588f08 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:28.518349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9a623-e79e-495c-98fd-b5ff20588f08 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:28.526080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9a623-e79e-495c-98fd-b5ff20588f08 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:28.529713 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9a623-e79e-495c-98fd-b5ff20588f08 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:28.534009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 79/157] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:28 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:28.536921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8285308-2196-4252-b27d-cf9456cd162a None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:28.537616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8285308-2196-4252-b27d-cf9456cd162a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:28.538256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8285308-2196-4252-b27d-cf9456cd162a None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:28.545310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 80/158] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:28 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:28.546873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf90abf-576a-4eaa-ba7e-6a949359f470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:28.547094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8285308-2196-4252-b27d-cf9456cd162a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:28.548861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf90abf-576a-4eaa-ba7e-6a949359f470 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:28.550083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf90abf-576a-4eaa-ba7e-6a949359f470 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:28.550744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf90abf-576a-4eaa-ba7e-6a949359f470 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:28.596129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf90abf-576a-4eaa-ba7e-6a949359f470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-caf90abf-576a-4eaa-ba7e-6a949359f470 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:28.611499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:28.612760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:28.613598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:28.617183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 79/159] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 492 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:28.622737 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c2fb5-f9d1-4da7-8ef3-de2192395345 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:28.623471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c2fb5-f9d1-4da7-8ef3-de2192395345 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:28.623951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c2fb5-f9d1-4da7-8ef3-de2192395345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:28.655559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f14c2fb5-f9d1-4da7-8ef3-de2192395345 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:28.657144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f14c2fb5-f9d1-4da7-8ef3-de2192395345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:28.787435 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8285308-2196-4252-b27d-cf9456cd162a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:28.832201 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635528fa-efc3-48d0-ad52-85b064712de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:28.833117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635528fa-efc3-48d0-ad52-85b064712de4 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:28.833582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635528fa-efc3-48d0-ad52-85b064712de4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:28.834170 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635528fa-efc3-48d0-ad52-85b064712de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:28.835352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635528fa-efc3-48d0-ad52-85b064712de4 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=8ff5716d18b14f8c83813becdf6695bd, user.id=a0034eb1f585466fae466bc41ea6f933) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:28.835814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635528fa-efc3-48d0-ad52-85b064712de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:28.838035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635528fa-efc3-48d0-ad52-85b064712de4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:28.846471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 81/160] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:28 2022] GET /identity/v3/role_assignments?user.id=a0034eb1f585466fae466bc41ea6f933&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:28.850723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635528fa-efc3-48d0-ad52-85b064712de4 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:28.852579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 80/161] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:28.964323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c2fb5-f9d1-4da7-8ef3-de2192395345 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:28.990769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635528fa-efc3-48d0-ad52-85b064712de4 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:29.027128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec016c-e3c7-421f-b1f2-e4c3ad40fe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:29.028132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec016c-e3c7-421f-b1f2-e4c3ad40fe26 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:29.028453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec016c-e3c7-421f-b1f2-e4c3ad40fe26 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:29.028692 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec016c-e3c7-421f-b1f2-e4c3ad40fe26 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:29.029750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec016c-e3c7-421f-b1f2-e4c3ad40fe26 None admin] RBAC: Adding query filter params (name=member) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:29.030195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec016c-e3c7-421f-b1f2-e4c3ad40fe26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:29.032770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec016c-e3c7-421f-b1f2-e4c3ad40fe26 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:29.043604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 82/162] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:29.049454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec016c-e3c7-421f-b1f2-e4c3ad40fe26 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:29.118371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c2fb5-f9d1-4da7-8ef3-de2192395345 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:29.184420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60cf62-8ccf-40a6-8ee4-76c27162d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:29.185271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60cf62-8ccf-40a6-8ee4-76c27162d3a8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:29.185546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60cf62-8ccf-40a6-8ee4-76c27162d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:29.185810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60cf62-8ccf-40a6-8ee4-76c27162d3a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:29.225266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60cf62-8ccf-40a6-8ee4-76c27162d3a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-bc60cf62-8ccf-40a6-8ee4-76c27162d3a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:29.249215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:29.250167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:29.251014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:29.252197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 81/163] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:40:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:29.265229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60cf62-8ccf-40a6-8ee4-76c27162d3a8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:29.313249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec016c-e3c7-421f-b1f2-e4c3ad40fe26 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:29.388097 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a5eb0-03f0-4f73-bfb1-f03c1ed3851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:29.390279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a5eb0-03f0-4f73-bfb1-f03c1ed3851a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:29.391066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a5eb0-03f0-4f73-bfb1-f03c1ed3851a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:29.391599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a5eb0-03f0-4f73-bfb1-f03c1ed3851a None admin] PATH_INFO: `/v3/users/adb0be7f832d47cfa9b23b4532305e94` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:29.393444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a5eb0-03f0-4f73-bfb1-f03c1ed3851a None admin] RBAC: Authorizing `identity:get_user(user_id=adb0be7f832d47cfa9b23b4532305e94)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:29.395957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a5eb0-03f0-4f73-bfb1-f03c1ed3851a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:29.400167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 83/164] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:29 2022] GET /identity/v3/users/adb0be7f832d47cfa9b23b4532305e94 => generated 275 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:29.422015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a5eb0-03f0-4f73-bfb1-f03c1ed3851a None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:29.543459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60cf62-8ccf-40a6-8ee4-76c27162d3a8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:29.640694 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f2384-10d3-4ea7-8765-4ed4d819eee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:29.641289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f2384-10d3-4ea7-8765-4ed4d819eee8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:29.641570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f2384-10d3-4ea7-8765-4ed4d819eee8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:29.641843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f2384-10d3-4ea7-8765-4ed4d819eee8 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:29.642614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f2384-10d3-4ea7-8765-4ed4d819eee8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:29.642942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f2384-10d3-4ea7-8765-4ed4d819eee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:29.644367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f2384-10d3-4ea7-8765-4ed4d819eee8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:29.657574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a5eb0-03f0-4f73-bfb1-f03c1ed3851a None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:29.658806 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 82/165] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:40:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:29.662910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3acb8-fefa-4a64-adcd-fbd8ed9f80d0 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:29.662910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3acb8-fefa-4a64-adcd-fbd8ed9f80d0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:29.662910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3acb8-fefa-4a64-adcd-fbd8ed9f80d0 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:29.670437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 83/166] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:29.673246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851ae27-1ed4-4801-8121-8510149620b7 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:29.673246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851ae27-1ed4-4801-8121-8510149620b7 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:29.673246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851ae27-1ed4-4801-8121-8510149620b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:29.710767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-c851ae27-1ed4-4801-8121-8510149620b7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:29.712513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c851ae27-1ed4-4801-8121-8510149620b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:29.723477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e769d0-0463-494e-9e7b-9e505a815f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:29.724892 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e769d0-0463-494e-9e7b-9e505a815f4b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:29.725189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e769d0-0463-494e-9e7b-9e505a815f4b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:29.725568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e769d0-0463-494e-9e7b-9e505a815f4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:29.874745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e769d0-0463-494e-9e7b-9e505a815f4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:29.876280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e769d0-0463-494e-9e7b-9e505a815f4b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:29.881556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 84/167] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:29.882462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851ae27-1ed4-4801-8121-8510149620b7 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:30.080499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851ae27-1ed4-4801-8121-8510149620b7 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:30.135590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a52ba9-11a0-460b-8bab-4d5b9dc3debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:30.136496 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a52ba9-11a0-460b-8bab-4d5b9dc3debc None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:30.136774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a52ba9-11a0-460b-8bab-4d5b9dc3debc None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:30.137047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a52ba9-11a0-460b-8bab-4d5b9dc3debc None admin] PATH_INFO: `/v3/users/adb0be7f832d47cfa9b23b4532305e94` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:30.138396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a52ba9-11a0-460b-8bab-4d5b9dc3debc None admin] RBAC: Authorizing `identity:get_user(user_id=adb0be7f832d47cfa9b23b4532305e94)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:30.139919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a52ba9-11a0-460b-8bab-4d5b9dc3debc None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:30.141948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 85/168] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:29 2022] GET /identity/v3/users/adb0be7f832d47cfa9b23b4532305e94 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:30.142797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a52ba9-11a0-460b-8bab-4d5b9dc3debc None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:30.153921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 84/169] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 732 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:40:30.154925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e769d0-0463-494e-9e7b-9e505a815f4b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:30.280740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a52ba9-11a0-460b-8bab-4d5b9dc3debc None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:30.333414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48b04b-4e2c-4441-a1ae-d568371098eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:30.335329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48b04b-4e2c-4441-a1ae-d568371098eb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:30.336175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48b04b-4e2c-4441-a1ae-d568371098eb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:30.337548 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48b04b-4e2c-4441-a1ae-d568371098eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:30.351338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e769d0-0463-494e-9e7b-9e505a815f4b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:30.362422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48b04b-4e2c-4441-a1ae-d568371098eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-df48b04b-4e2c-4441-a1ae-d568371098eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:30.374366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:30.376442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:30.378480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:30.380678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 86/170] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:30.381669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48b04b-4e2c-4441-a1ae-d568371098eb None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:30.412664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3b18f-dc3e-4984-a0e1-f6ac1aa77662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:30.413659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3b18f-dc3e-4984-a0e1-f6ac1aa77662 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:30.414124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3b18f-dc3e-4984-a0e1-f6ac1aa77662 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:30.414804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3b18f-dc3e-4984-a0e1-f6ac1aa77662 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:30.517560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3b18f-dc3e-4984-a0e1-f6ac1aa77662 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:30.520081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3b18f-dc3e-4984-a0e1-f6ac1aa77662 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:30.558851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48b04b-4e2c-4441-a1ae-d568371098eb None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:30.618408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9ce85-bdce-455e-99c8-98345433f185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:30.620207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9ce85-bdce-455e-99c8-98345433f185 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:30.620766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9ce85-bdce-455e-99c8-98345433f185 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:30.621227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9ce85-bdce-455e-99c8-98345433f185 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:30.623455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9ce85-bdce-455e-99c8-98345433f185 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:30.625802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9ce85-bdce-455e-99c8-98345433f185 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:30.628589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 87/171] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:30 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:30.629921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9ce85-bdce-455e-99c8-98345433f185 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:30.676301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 85/172] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:30.677495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3b18f-dc3e-4984-a0e1-f6ac1aa77662 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:30.797574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3b18f-dc3e-4984-a0e1-f6ac1aa77662 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:30.797836 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9ce85-bdce-455e-99c8-98345433f185 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:30.841706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa855537-8021-48a6-b53a-0351e3f9174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:30.843003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa855537-8021-48a6-b53a-0351e3f9174c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:30.843686 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa855537-8021-48a6-b53a-0351e3f9174c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:30.844290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa855537-8021-48a6-b53a-0351e3f9174c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:30.846003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa855537-8021-48a6-b53a-0351e3f9174c None admin] RBAC: Adding query filter params (role.id=90868071f00340ae853f1188777a2244, scope.project.id=8ff5716d18b14f8c83813becdf6695bd, user.id=adb0be7f832d47cfa9b23b4532305e94) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:30.846932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa855537-8021-48a6-b53a-0351e3f9174c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:30.849883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa855537-8021-48a6-b53a-0351e3f9174c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:30.861701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 86/173] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:30 2022] GET /identity/v3/role_assignments?user.id=adb0be7f832d47cfa9b23b4532305e94&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=90868071f00340ae853f1188777a2244 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:30.863690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa855537-8021-48a6-b53a-0351e3f9174c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:30.897727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ec67b-bcd4-4be7-a3cf-071f0d7b9fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:30.902034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ec67b-bcd4-4be7-a3cf-071f0d7b9fcb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:30.902246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ec67b-bcd4-4be7-a3cf-071f0d7b9fcb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:30.902657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ec67b-bcd4-4be7-a3cf-071f0d7b9fcb None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:30.911499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ec67b-bcd4-4be7-a3cf-071f0d7b9fcb None admin] RBAC: Adding query filter params (name=member) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:30.912072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ec67b-bcd4-4be7-a3cf-071f0d7b9fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:30.914646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ec67b-bcd4-4be7-a3cf-071f0d7b9fcb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:30.952707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 88/174] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:30.960481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ec67b-bcd4-4be7-a3cf-071f0d7b9fcb None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:31.161775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa855537-8021-48a6-b53a-0351e3f9174c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:31.205223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cbe5d-fd9b-48bd-81ef-2604e88316c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:31.206585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cbe5d-fd9b-48bd-81ef-2604e88316c4 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.207223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cbe5d-fd9b-48bd-81ef-2604e88316c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.207729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cbe5d-fd9b-48bd-81ef-2604e88316c4 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.209943 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cbe5d-fd9b-48bd-81ef-2604e88316c4 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:31.211940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cbe5d-fd9b-48bd-81ef-2604e88316c4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:31.212480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ec67b-bcd4-4be7-a3cf-071f0d7b9fcb None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:31.214469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 87/175] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:30 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:31.216229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fb573-9adc-4fd9-9be8-a5ffeeaf2023 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.216734 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fb573-9adc-4fd9-9be8-a5ffeeaf2023 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.217230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fb573-9adc-4fd9-9be8-a5ffeeaf2023 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.218876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 88/176] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:31.227144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0180f08-a1bf-4989-afff-06822923ecc8 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.227768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0180f08-a1bf-4989-afff-06822923ecc8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.228248 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0180f08-a1bf-4989-afff-06822923ecc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.247029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a0180f08-a1bf-4989-afff-06822923ecc8 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:31.248621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a0180f08-a1bf-4989-afff-06822923ecc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:31.271436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54419767-78f5-4525-9485-35f4d7071dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:31.272335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54419767-78f5-4525-9485-35f4d7071dde None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.272612 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54419767-78f5-4525-9485-35f4d7071dde None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.272882 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54419767-78f5-4525-9485-35f4d7071dde None admin] PATH_INFO: `/v3/users/adb0be7f832d47cfa9b23b4532305e94` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.274238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54419767-78f5-4525-9485-35f4d7071dde None admin] RBAC: Authorizing `identity:get_user(user_id=adb0be7f832d47cfa9b23b4532305e94)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:31.275772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54419767-78f5-4525-9485-35f4d7071dde None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:31.277864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 89/177] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:30 2022] GET /identity/v3/users/adb0be7f832d47cfa9b23b4532305e94 => generated 275 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:31.279224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54419767-78f5-4525-9485-35f4d7071dde None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:31.333878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 89/178] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:31.335215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0180f08-a1bf-4989-afff-06822923ecc8 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:31.389798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54419767-78f5-4525-9485-35f4d7071dde None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:31.426589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bc967-7cc3-474e-89bd-e967ab24063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:31.427487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bc967-7cc3-474e-89bd-e967ab24063f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.427822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bc967-7cc3-474e-89bd-e967ab24063f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.428099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bc967-7cc3-474e-89bd-e967ab24063f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.428957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bc967-7cc3-474e-89bd-e967ab24063f None admin] RBAC: Adding query filter params (role.id=4501501962414ed0abfba693ba95918c, scope.project.id=8ff5716d18b14f8c83813becdf6695bd, user.id=adb0be7f832d47cfa9b23b4532305e94) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:31.429251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bc967-7cc3-474e-89bd-e967ab24063f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:31.431404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bc967-7cc3-474e-89bd-e967ab24063f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:31.437799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 90/179] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:31 2022] GET /identity/v3/role_assignments?user.id=adb0be7f832d47cfa9b23b4532305e94&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=4501501962414ed0abfba693ba95918c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:31.438654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bc967-7cc3-474e-89bd-e967ab24063f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:31.452873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0180f08-a1bf-4989-afff-06822923ecc8 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:31.493883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809dd0-a802-47c2-852a-805dc8fc7879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:31.495047 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809dd0-a802-47c2-852a-805dc8fc7879 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.495542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809dd0-a802-47c2-852a-805dc8fc7879 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.496015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809dd0-a802-47c2-852a-805dc8fc7879 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.560927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bc967-7cc3-474e-89bd-e967ab24063f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:31.577748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76809dd0-a802-47c2-852a-805dc8fc7879 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:31.579974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76809dd0-a802-47c2-852a-805dc8fc7879 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:31.601094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4dbd91-19b6-47c4-ad5f-2aafccfd61f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:31.602605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4dbd91-19b6-47c4-ad5f-2aafccfd61f4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.602825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4dbd91-19b6-47c4-ad5f-2aafccfd61f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.603057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4dbd91-19b6-47c4-ad5f-2aafccfd61f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.617212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4dbd91-19b6-47c4-ad5f-2aafccfd61f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-7f4dbd91-19b6-47c4-ad5f-2aafccfd61f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:31.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:31.626798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:31.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 91/180] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:40:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:31.628773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a4241-e8ee-4969-aa5b-63376413149a None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.629010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a4241-e8ee-4969-aa5b-63376413149a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.629278 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a4241-e8ee-4969-aa5b-63376413149a None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.630929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 92/181] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:31.637350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68b68c-451a-4277-8394-c7ac915564b9 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.637681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68b68c-451a-4277-8394-c7ac915564b9 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.637901 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68b68c-451a-4277-8394-c7ac915564b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.656276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-5d68b68c-451a-4277-8394-c7ac915564b9 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:31.657665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5d68b68c-451a-4277-8394-c7ac915564b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:31.710859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 90/182] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:31.712129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809dd0-a802-47c2-852a-805dc8fc7879 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:31.739700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 93/183] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:31.740938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68b68c-451a-4277-8394-c7ac915564b9 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:31.837171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809dd0-a802-47c2-852a-805dc8fc7879 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:31.877473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fad271-a912-4519-a003-c4be04ae4670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:31.878636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fad271-a912-4519-a003-c4be04ae4670 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.879298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fad271-a912-4519-a003-c4be04ae4670 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.879828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fad271-a912-4519-a003-c4be04ae4670 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.881061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fad271-a912-4519-a003-c4be04ae4670 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:31.881629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fad271-a912-4519-a003-c4be04ae4670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:31.883937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fad271-a912-4519-a003-c4be04ae4670 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:31.888226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68b68c-451a-4277-8394-c7ac915564b9 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:31.894859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 91/184] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:31.896272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fad271-a912-4519-a003-c4be04ae4670 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:31.935356 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281eb5c-93c7-46f6-a42e-e9d7212a24e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:31.936255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281eb5c-93c7-46f6-a42e-e9d7212a24e9 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:31.936533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281eb5c-93c7-46f6-a42e-e9d7212a24e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:31.936804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281eb5c-93c7-46f6-a42e-e9d7212a24e9 None admin] PATH_INFO: `/v3/projects/4e1638ccc38b47e39a1be33df7681a22` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:31.938634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2281eb5c-93c7-46f6-a42e-e9d7212a24e9 None admin] RBAC: Authorizing `identity:get_project(project_id=4e1638ccc38b47e39a1be33df7681a22)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:31.939895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2281eb5c-93c7-46f6-a42e-e9d7212a24e9 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:31.942206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 94/185] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:31 2022] GET /identity/v3/projects/4e1638ccc38b47e39a1be33df7681a22 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:31.943980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281eb5c-93c7-46f6-a42e-e9d7212a24e9 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:32.096552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fad271-a912-4519-a003-c4be04ae4670 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:32.114258 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2281eb5c-93c7-46f6-a42e-e9d7212a24e9 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:32.155737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdf3bc-d906-471d-b8c2-ed7d57659651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:32.157194 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdf3bc-d906-471d-b8c2-ed7d57659651 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:32.157793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdf3bc-d906-471d-b8c2-ed7d57659651 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:32.158444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdf3bc-d906-471d-b8c2-ed7d57659651 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:32.172918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdf3bc-d906-471d-b8c2-ed7d57659651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:32.179350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4069d-160c-42fb-87cc-a6e65a25a025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:32.181304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4069d-160c-42fb-87cc-a6e65a25a025 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:32.181763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4069d-160c-42fb-87cc-a6e65a25a025 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:32.182208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4069d-160c-42fb-87cc-a6e65a25a025 None admin] PATH_INFO: `/v3/users/22d5df8d31334c9581ecdecb52478cac` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-cbfdf3bc-d906-471d-b8c2-ed7d57659651 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:32.185743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:32.186651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:32.187531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:32.190594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 92/186] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:40:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:32.191876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdf3bc-d906-471d-b8c2-ed7d57659651 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:32.217489 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4069d-160c-42fb-87cc-a6e65a25a025 None admin] RBAC: Authorizing `identity:get_user(user_id=22d5df8d31334c9581ecdecb52478cac)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:32.220168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4069d-160c-42fb-87cc-a6e65a25a025 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:32.223673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 95/187] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:31 2022] GET /identity/v3/users/22d5df8d31334c9581ecdecb52478cac => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:32.225344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4069d-160c-42fb-87cc-a6e65a25a025 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:32.328837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdf3bc-d906-471d-b8c2-ed7d57659651 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:32.374611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f42c8f-4ccf-4cb2-9b70-b659e40a97d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:32.375837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42c8f-4ccf-4cb2-9b70-b659e40a97d7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:32.376343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42c8f-4ccf-4cb2-9b70-b659e40a97d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:32.376847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42c8f-4ccf-4cb2-9b70-b659e40a97d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:32.378556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f42c8f-4ccf-4cb2-9b70-b659e40a97d7 None admin] RBAC: Adding query filter params (role.id=90868071f00340ae853f1188777a2244, scope.project.id=4e1638ccc38b47e39a1be33df7681a22, user.id=adb0be7f832d47cfa9b23b4532305e94) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:32.379161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f42c8f-4ccf-4cb2-9b70-b659e40a97d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:32.382095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f42c8f-4ccf-4cb2-9b70-b659e40a97d7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:32.384799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4069d-160c-42fb-87cc-a6e65a25a025 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:32.393459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 93/188] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:32 2022] GET /identity/v3/role_assignments?user.id=adb0be7f832d47cfa9b23b4532305e94&scope.project.id=4e1638ccc38b47e39a1be33df7681a22&role.id=90868071f00340ae853f1188777a2244 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:32.393459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dd3a1-bd38-4133-912c-f48f88419f31 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:32.393459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dd3a1-bd38-4133-912c-f48f88419f31 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:32.393459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503dd3a1-bd38-4133-912c-f48f88419f31 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:32.393459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 94/189] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:32.400616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503dd3a1-bd38-4133-912c-f48f88419f31 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:32.425318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00346546-9fff-456e-9128-c5e08680b61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:32.426015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00346546-9fff-456e-9128-c5e08680b61b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:32.426296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00346546-9fff-456e-9128-c5e08680b61b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:32.426568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00346546-9fff-456e-9128-c5e08680b61b None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:32.427508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00346546-9fff-456e-9128-c5e08680b61b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:32.427925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00346546-9fff-456e-9128-c5e08680b61b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:32.429401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00346546-9fff-456e-9128-c5e08680b61b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:32.439888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 96/190] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:40:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:32.441069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f08db-2f86-494e-8372-4d1aa06a399f None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:32.441345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f08db-2f86-494e-8372-4d1aa06a399f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:32.441619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f08db-2f86-494e-8372-4d1aa06a399f None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:32.446776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 97/191] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:32.446776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59012f9c-c736-4e06-ad22-ab4eee0d2953 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:32.446776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59012f9c-c736-4e06-ad22-ab4eee0d2953 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:32.446776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59012f9c-c736-4e06-ad22-ab4eee0d2953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:32.464810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-59012f9c-c736-4e06-ad22-ab4eee0d2953 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:32.466058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-59012f9c-c736-4e06-ad22-ab4eee0d2953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:32.550644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 98/192] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:32.553558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503dd3a1-bd38-4133-912c-f48f88419f31 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:32.556956 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59012f9c-c736-4e06-ad22-ab4eee0d2953 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:32.611769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3547d5-baf7-4f69-a1df-dd0d0da071ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:32.612332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3547d5-baf7-4f69-a1df-dd0d0da071ab None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:32.612610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3547d5-baf7-4f69-a1df-dd0d0da071ab None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:32.612891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3547d5-baf7-4f69-a1df-dd0d0da071ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:32.729816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59012f9c-c736-4e06-ad22-ab4eee0d2953 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:32.734165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3547d5-baf7-4f69-a1df-dd0d0da071ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:32.735742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3547d5-baf7-4f69-a1df-dd0d0da071ab None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:32.794124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38909986-ffbe-4694-ac8f-050d59010d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:32.795610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38909986-ffbe-4694-ac8f-050d59010d4a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:32.796004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38909986-ffbe-4694-ac8f-050d59010d4a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:32.796507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38909986-ffbe-4694-ac8f-050d59010d4a None admin] PATH_INFO: `/v3/users/22d5df8d31334c9581ecdecb52478cac` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:32.798814 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38909986-ffbe-4694-ac8f-050d59010d4a None admin] RBAC: Authorizing `identity:get_user(user_id=22d5df8d31334c9581ecdecb52478cac)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:32.801659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38909986-ffbe-4694-ac8f-050d59010d4a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:32.809184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 99/193] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:32 2022] GET /identity/v3/users/22d5df8d31334c9581ecdecb52478cac => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:32.809184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853947c6-630e-45a7-a7b2-10a2675558fb None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:32.809184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853947c6-630e-45a7-a7b2-10a2675558fb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:32.809184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853947c6-630e-45a7-a7b2-10a2675558fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:32.833223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-853947c6-630e-45a7-a7b2-10a2675558fb None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:32.835431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-853947c6-630e-45a7-a7b2-10a2675558fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:32.861597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 95/194] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:32.862993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3547d5-baf7-4f69-a1df-dd0d0da071ab None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:32.925033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 100/195] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:32.925913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853947c6-630e-45a7-a7b2-10a2675558fb None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:32.996749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3547d5-baf7-4f69-a1df-dd0d0da071ab None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:33.040910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d740236-c62d-4e7c-98e9-52f3b216aa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:33.041728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d740236-c62d-4e7c-98e9-52f3b216aa16 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:33.042010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d740236-c62d-4e7c-98e9-52f3b216aa16 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:33.042284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d740236-c62d-4e7c-98e9-52f3b216aa16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:33.043671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d740236-c62d-4e7c-98e9-52f3b216aa16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:33.045200 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d740236-c62d-4e7c-98e9-52f3b216aa16 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:33.047312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 96/196] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:33.047918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d740236-c62d-4e7c-98e9-52f3b216aa16 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:33.069794 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853947c6-630e-45a7-a7b2-10a2675558fb None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:33.114952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c4181-3910-4c22-b756-4af8b3bc431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:33.115771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c4181-3910-4c22-b756-4af8b3bc431a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:33.116323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c4181-3910-4c22-b756-4af8b3bc431a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:33.116765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c4181-3910-4c22-b756-4af8b3bc431a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:33.179747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d740236-c62d-4e7c-98e9-52f3b216aa16 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:33.223780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca2c89-07ed-49ba-83e8-f42ceb9e030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:33.224707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca2c89-07ed-49ba-83e8-f42ceb9e030d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:33.224986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca2c89-07ed-49ba-83e8-f42ceb9e030d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:33.225292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca2c89-07ed-49ba-83e8-f42ceb9e030d None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:33.226609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca2c89-07ed-49ba-83e8-f42ceb9e030d None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:33.228158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca2c89-07ed-49ba-83e8-f42ceb9e030d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:33.229725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c4181-3910-4c22-b756-4af8b3bc431a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:33.236869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 97/197] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:33 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:33.237032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c4181-3910-4c22-b756-4af8b3bc431a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:33.237169 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca2c89-07ed-49ba-83e8-f42ceb9e030d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:33.386126 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 101/198] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:33.387561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c4181-3910-4c22-b756-4af8b3bc431a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:33.387942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca2c89-07ed-49ba-83e8-f42ceb9e030d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:33.433867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df8ef4-a27a-43a9-845f-fc652978db95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:33.435080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8ef4-a27a-43a9-845f-fc652978db95 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:33.435431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8ef4-a27a-43a9-845f-fc652978db95 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:33.435774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8ef4-a27a-43a9-845f-fc652978db95 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:33.449506 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df8ef4-a27a-43a9-845f-fc652978db95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-96df8ef4-a27a-43a9-845f-fc652978db95 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:33.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:33.459384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 98/199] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:33.460753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df8ef4-a27a-43a9-845f-fc652978db95 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:33.535364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c4181-3910-4c22-b756-4af8b3bc431a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:33.590952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841d37b-a52b-4daa-b018-1ae2df29fab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:33.592051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841d37b-a52b-4daa-b018-1ae2df29fab8 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:33.592482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841d37b-a52b-4daa-b018-1ae2df29fab8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:33.592912 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841d37b-a52b-4daa-b018-1ae2df29fab8 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:33.594142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841d37b-a52b-4daa-b018-1ae2df29fab8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:33.596666 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841d37b-a52b-4daa-b018-1ae2df29fab8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:33.624143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df8ef4-a27a-43a9-845f-fc652978db95 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:33.624692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 102/200] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:40:33 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:40:33.639112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841d37b-a52b-4daa-b018-1ae2df29fab8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:33.668984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f348f60-684d-4be6-893e-885cf4a6cfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:33.670172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f348f60-684d-4be6-893e-885cf4a6cfa4 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:33.670476 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f348f60-684d-4be6-893e-885cf4a6cfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:33.673396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f348f60-684d-4be6-893e-885cf4a6cfa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:33.673396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f348f60-684d-4be6-893e-885cf4a6cfa4 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=e2682c48ad134e689170645b2c3b85de, user.id=22d5df8d31334c9581ecdecb52478cac) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:33.673396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f348f60-684d-4be6-893e-885cf4a6cfa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:33.673837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f348f60-684d-4be6-893e-885cf4a6cfa4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:33.694748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 99/201] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:33 2022] GET /identity/v3/role_assignments?user.id=22d5df8d31334c9581ecdecb52478cac&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:33.694748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f348f60-684d-4be6-893e-885cf4a6cfa4 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:33.787657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841d37b-a52b-4daa-b018-1ae2df29fab8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:33.829596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f9962-975c-4a8c-92cb-fd10921d5d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:33.834751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f9962-975c-4a8c-92cb-fd10921d5d79 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:33.834751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f9962-975c-4a8c-92cb-fd10921d5d79 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:33.834751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f9962-975c-4a8c-92cb-fd10921d5d79 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:33.834751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f9962-975c-4a8c-92cb-fd10921d5d79 None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:33.835414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f348f60-684d-4be6-893e-885cf4a6cfa4 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:33.835556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f9962-975c-4a8c-92cb-fd10921d5d79 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:33.838304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 103/202] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:33 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:33.845279 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f9962-975c-4a8c-92cb-fd10921d5d79 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:33.900001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538dea1-bb20-498e-a1b1-bebb54ff52e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:33.900360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538dea1-bb20-498e-a1b1-bebb54ff52e6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:33.900759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538dea1-bb20-498e-a1b1-bebb54ff52e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:33.901140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538dea1-bb20-498e-a1b1-bebb54ff52e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:33.902202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538dea1-bb20-498e-a1b1-bebb54ff52e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:33.902532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538dea1-bb20-498e-a1b1-bebb54ff52e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:33.904887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538dea1-bb20-498e-a1b1-bebb54ff52e6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:33.918774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 100/203] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:33.928377 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538dea1-bb20-498e-a1b1-bebb54ff52e6 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:34.009725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f9962-975c-4a8c-92cb-fd10921d5d79 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:34.053884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538dea1-bb20-498e-a1b1-bebb54ff52e6 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:34.054089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc9daa-9190-4778-bfb9-736006ab1fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:34.054825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc9daa-9190-4778-bfb9-736006ab1fc1 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:34.055131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc9daa-9190-4778-bfb9-736006ab1fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:34.055405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc9daa-9190-4778-bfb9-736006ab1fc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:34.056589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc9daa-9190-4778-bfb9-736006ab1fc1 None admin] RBAC: Adding query filter params (role.id=4501501962414ed0abfba693ba95918c, scope.project.id=e2682c48ad134e689170645b2c3b85de, user.id=22d5df8d31334c9581ecdecb52478cac) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:34.056909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc9daa-9190-4778-bfb9-736006ab1fc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:34.058431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc9daa-9190-4778-bfb9-736006ab1fc1 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:34.065136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 104/204] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:33 2022] GET /identity/v3/role_assignments?user.id=22d5df8d31334c9581ecdecb52478cac&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=4501501962414ed0abfba693ba95918c => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:34.108088 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71ba5f-d388-4d7b-8946-29e99bbe8507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:34.109721 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71ba5f-d388-4d7b-8946-29e99bbe8507 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:34.110262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71ba5f-d388-4d7b-8946-29e99bbe8507 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:34.110826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71ba5f-d388-4d7b-8946-29e99bbe8507 None admin] PATH_INFO: `/v3/users/5cf1ec9c433941228e63ec46cbc595eb` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:34.112597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71ba5f-d388-4d7b-8946-29e99bbe8507 None admin] RBAC: Authorizing `identity:get_user(user_id=5cf1ec9c433941228e63ec46cbc595eb)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:34.114377 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71ba5f-d388-4d7b-8946-29e99bbe8507 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:34.116764 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 101/205] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:33 2022] GET /identity/v3/users/5cf1ec9c433941228e63ec46cbc595eb => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:34.126794 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc9daa-9190-4778-bfb9-736006ab1fc1 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:34.284533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc9daa-9190-4778-bfb9-736006ab1fc1 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:34.351652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcda7f0-ff0e-4d30-8024-7590f4721506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:34.353383 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcda7f0-ff0e-4d30-8024-7590f4721506 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:34.353956 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcda7f0-ff0e-4d30-8024-7590f4721506 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:34.354445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcda7f0-ff0e-4d30-8024-7590f4721506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:34.485374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcda7f0-ff0e-4d30-8024-7590f4721506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:34.488128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcda7f0-ff0e-4d30-8024-7590f4721506 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:34.679005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 105/206] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:34.684972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71ba5f-d388-4d7b-8946-29e99bbe8507 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:34.871407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71ba5f-d388-4d7b-8946-29e99bbe8507 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:34.926543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1c7f8-b3d1-488d-b2f0-ff5081d635c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:34.927829 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1c7f8-b3d1-488d-b2f0-ff5081d635c6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:34.928318 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1c7f8-b3d1-488d-b2f0-ff5081d635c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:34.928794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1c7f8-b3d1-488d-b2f0-ff5081d635c6 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:34.933911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1c7f8-b3d1-488d-b2f0-ff5081d635c6 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:34.935729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1c7f8-b3d1-488d-b2f0-ff5081d635c6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:34.938079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 102/207] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:34 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:34.947248 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcda7f0-ff0e-4d30-8024-7590f4721506 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:35.121947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcda7f0-ff0e-4d30-8024-7590f4721506 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:35.171670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd280dfd-2b85-4bfa-a849-5430ba2d3067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:35.172307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd280dfd-2b85-4bfa-a849-5430ba2d3067 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:35.172632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd280dfd-2b85-4bfa-a849-5430ba2d3067 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:35.172901 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd280dfd-2b85-4bfa-a849-5430ba2d3067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:35.173802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd280dfd-2b85-4bfa-a849-5430ba2d3067 None admin] RBAC: Adding query filter params (role.id=a00af3df784a4cbf9d4a59157edebd16, scope.project.id=8ff5716d18b14f8c83813becdf6695bd, user.id=5cf1ec9c433941228e63ec46cbc595eb) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:35.174202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd280dfd-2b85-4bfa-a849-5430ba2d3067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:35.175833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd280dfd-2b85-4bfa-a849-5430ba2d3067 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:35.190970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 106/208] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:34 2022] GET /identity/v3/role_assignments?user.id=5cf1ec9c433941228e63ec46cbc595eb&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=a00af3df784a4cbf9d4a59157edebd16 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:36.771543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413e0e4-60cb-495a-ad53-01bfadf6c10a None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:36.773239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413e0e4-60cb-495a-ad53-01bfadf6c10a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:36.774384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413e0e4-60cb-495a-ad53-01bfadf6c10a None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:36.776141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 103/209] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:36.782333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce33fb2-613d-47e5-995d-63fdd515412d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:36.783488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce33fb2-613d-47e5-995d-63fdd515412d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:36.784006 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce33fb2-613d-47e5-995d-63fdd515412d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:36.803784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-dce33fb2-613d-47e5-995d-63fdd515412d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:36.805514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-dce33fb2-613d-47e5-995d-63fdd515412d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:36.887019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 107/210] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:36.910534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413e0e4-60cb-495a-ad53-01bfadf6c10a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:37.064451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413e0e4-60cb-495a-ad53-01bfadf6c10a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:37.094323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dbac4-e9dd-461e-997c-86131f9facf4 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.094841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dbac4-e9dd-461e-997c-86131f9facf4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.095331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dbac4-e9dd-461e-997c-86131f9facf4 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.097908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 108/211] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:40:37.108214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c59ab-c8c0-4bd6-a020-69e4d6c0a252 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.109173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c59ab-c8c0-4bd6-a020-69e4d6c0a252 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.109925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c59ab-c8c0-4bd6-a020-69e4d6c0a252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.116211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da8907-7b6f-4ffa-8687-6bb003f504a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:37.117958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8907-7b6f-4ffa-8687-6bb003f504a7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.118328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8907-7b6f-4ffa-8687-6bb003f504a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.118629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8907-7b6f-4ffa-8687-6bb003f504a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.135997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-769c59ab-c8c0-4bd6-a020-69e4d6c0a252 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:37.136218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da8907-7b6f-4ffa-8687-6bb003f504a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:37.137719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-769c59ab-c8c0-4bd6-a020-69e4d6c0a252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-54da8907-7b6f-4ffa-8687-6bb003f504a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:37.151600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:37.152643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:37.153899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:37.154565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 104/212] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:40:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:37.163492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da8907-7b6f-4ffa-8687-6bb003f504a7 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:37.216844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 109/213] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:37.241522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c59ab-c8c0-4bd6-a020-69e4d6c0a252 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:37.296638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da8907-7b6f-4ffa-8687-6bb003f504a7 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:37.337843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef106f4-17c8-4c07-b12a-b9a0c00f93b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:37.338890 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef106f4-17c8-4c07-b12a-b9a0c00f93b3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.339698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef106f4-17c8-4c07-b12a-b9a0c00f93b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.339896 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef106f4-17c8-4c07-b12a-b9a0c00f93b3 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.341167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef106f4-17c8-4c07-b12a-b9a0c00f93b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:37.341702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef106f4-17c8-4c07-b12a-b9a0c00f93b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:37.344329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef106f4-17c8-4c07-b12a-b9a0c00f93b3 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:37.359721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c59ab-c8c0-4bd6-a020-69e4d6c0a252 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:37.365305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 105/214] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:37.365305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9e512-1e63-465d-8aec-58ec01902e98 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.365305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9e512-1e63-465d-8aec-58ec01902e98 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.365305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9e512-1e63-465d-8aec-58ec01902e98 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.366381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 106/215] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:37.369053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9e512-1e63-465d-8aec-58ec01902e98 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:37.426130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea8819-31d4-42be-bc6d-f2841c7d0b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:37.427505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8819-31d4-42be-bc6d-f2841c7d0b00 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.427987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8819-31d4-42be-bc6d-f2841c7d0b00 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.428439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8819-31d4-42be-bc6d-f2841c7d0b00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.451708 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea8819-31d4-42be-bc6d-f2841c7d0b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-92ea8819-31d4-42be-bc6d-f2841c7d0b00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:37.464236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:37.465304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:37.466085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:37.473062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 110/216] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:40:37.473062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b1ea2-b9b1-4bbe-988b-4236455d4f7c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.473062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b1ea2-b9b1-4bbe-988b-4236455d4f7c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.473062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b1ea2-b9b1-4bbe-988b-4236455d4f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.492547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e40b1ea2-b9b1-4bbe-988b-4236455d4f7c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:37.493779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e40b1ea2-b9b1-4bbe-988b-4236455d4f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:37.530178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9e512-1e63-465d-8aec-58ec01902e98 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:37.573157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23387f8e-96ca-4749-88f5-351dc9f72af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:37.586744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 111/217] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:37.587052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23387f8e-96ca-4749-88f5-351dc9f72af9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.587052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23387f8e-96ca-4749-88f5-351dc9f72af9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.587052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23387f8e-96ca-4749-88f5-351dc9f72af9 None admin] PATH_INFO: `/v3/users/22d5df8d31334c9581ecdecb52478cac` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.587052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23387f8e-96ca-4749-88f5-351dc9f72af9 None admin] RBAC: Authorizing `identity:get_user(user_id=22d5df8d31334c9581ecdecb52478cac)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:37.587587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5fcf5-cc64-48c6-9aef-f77b60d813c3 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.587587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5fcf5-cc64-48c6-9aef-f77b60d813c3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.587587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5fcf5-cc64-48c6-9aef-f77b60d813c3 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.587995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23387f8e-96ca-4749-88f5-351dc9f72af9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:37.587995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 107/218] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:37 2022] GET /identity/v3/users/22d5df8d31334c9581ecdecb52478cac => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:37.588276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 112/219] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:37.588412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23387f8e-96ca-4749-88f5-351dc9f72af9 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:37.590821 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5fcf5-cc64-48c6-9aef-f77b60d813c3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:37.736885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23387f8e-96ca-4749-88f5-351dc9f72af9 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:37.745691 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5fcf5-cc64-48c6-9aef-f77b60d813c3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:37.797248 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450bf5f-b11f-466a-90b4-7f1aaad404b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:37.798261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450bf5f-b11f-466a-90b4-7f1aaad404b3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.798730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450bf5f-b11f-466a-90b4-7f1aaad404b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.799188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450bf5f-b11f-466a-90b4-7f1aaad404b3 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.800530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450bf5f-b11f-466a-90b4-7f1aaad404b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:37.801037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450bf5f-b11f-466a-90b4-7f1aaad404b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:37.803356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450bf5f-b11f-466a-90b4-7f1aaad404b3 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:37.807305 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c767c9b-683b-4528-9810-16d24d686787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:37.808839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767c9b-683b-4528-9810-16d24d686787 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.809338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767c9b-683b-4528-9810-16d24d686787 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.809748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767c9b-683b-4528-9810-16d24d686787 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.812048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c767c9b-683b-4528-9810-16d24d686787 None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:37.813468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 108/220] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:37.814614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eb10a-3b54-4a08-b9c7-8c86da2eb3d7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:37.814935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eb10a-3b54-4a08-b9c7-8c86da2eb3d7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:37.815257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c767c9b-683b-4528-9810-16d24d686787 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:37.815431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eb10a-3b54-4a08-b9c7-8c86da2eb3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:37.822437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 113/221] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:37 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:37.822437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c767c9b-683b-4528-9810-16d24d686787 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:37.846860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-161eb10a-3b54-4a08-b9c7-8c86da2eb3d7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:37.848009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-161eb10a-3b54-4a08-b9c7-8c86da2eb3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:37.974768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 109/222] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:37.974768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161eb10a-3b54-4a08-b9c7-8c86da2eb3d7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:37.980441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c767c9b-683b-4528-9810-16d24d686787 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:38.026314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b34eb7-aebb-403b-90c3-3e5f5239889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:38.027264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b34eb7-aebb-403b-90c3-3e5f5239889d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:38.027610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b34eb7-aebb-403b-90c3-3e5f5239889d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:38.027883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b34eb7-aebb-403b-90c3-3e5f5239889d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:38.044446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b34eb7-aebb-403b-90c3-3e5f5239889d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-43b34eb7-aebb-403b-90c3-3e5f5239889d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:38.052872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:38.055642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:38.057701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 114/223] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:40:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:38.061235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b34eb7-aebb-403b-90c3-3e5f5239889d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:38.132485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161eb10a-3b54-4a08-b9c7-8c86da2eb3d7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:38.187049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36be2b-d2a5-47a5-9b67-3f8db5479beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:38.188431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36be2b-d2a5-47a5-9b67-3f8db5479beb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:38.188807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36be2b-d2a5-47a5-9b67-3f8db5479beb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:38.189158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36be2b-d2a5-47a5-9b67-3f8db5479beb None admin] PATH_INFO: `/v3/users/9aa5e833826847db823c536f5b5e59ef` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:38.197801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b34eb7-aebb-403b-90c3-3e5f5239889d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:38.214458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36be2b-d2a5-47a5-9b67-3f8db5479beb None admin] RBAC: Authorizing `identity:get_user(user_id=9aa5e833826847db823c536f5b5e59ef)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:38.215983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36be2b-d2a5-47a5-9b67-3f8db5479beb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:38.217719 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 110/224] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:37 2022] GET /identity/v3/users/9aa5e833826847db823c536f5b5e59ef => generated 297 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:38.219563 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36be2b-d2a5-47a5-9b67-3f8db5479beb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:38.249820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0098c3-3e3d-47e5-a946-5043abd5e100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:38.250752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0098c3-3e3d-47e5-a946-5043abd5e100 None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:38.251087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0098c3-3e3d-47e5-a946-5043abd5e100 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:38.251442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0098c3-3e3d-47e5-a946-5043abd5e100 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de/users/22d5df8d31334c9581ecdecb52478cac/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:38.253555 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0098c3-3e3d-47e5-a946-5043abd5e100 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2682c48ad134e689170645b2c3b85de, user_id=22d5df8d31334c9581ecdecb52478cac, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:38.255115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0098c3-3e3d-47e5-a946-5043abd5e100 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:38.272100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 115/225] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:40:38 2022] PUT /identity/v3/projects/e2682c48ad134e689170645b2c3b85de/users/22d5df8d31334c9581ecdecb52478cac/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:40:38.272958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0098c3-3e3d-47e5-a946-5043abd5e100 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:38.361217 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36be2b-d2a5-47a5-9b67-3f8db5479beb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:38.416849 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0098c3-3e3d-47e5-a946-5043abd5e100 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:38.420149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c67c9-bafa-4e55-9b4a-1508a79b258d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:38.421603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c67c9-bafa-4e55-9b4a-1508a79b258d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:38.422087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c67c9-bafa-4e55-9b4a-1508a79b258d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:38.422568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c67c9-bafa-4e55-9b4a-1508a79b258d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:38.424231 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c67c9-bafa-4e55-9b4a-1508a79b258d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:38.425921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c67c9-bafa-4e55-9b4a-1508a79b258d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:38.430807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 111/226] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:38.430807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c67c9-bafa-4e55-9b4a-1508a79b258d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:38.463133 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09001e57-cf29-48dd-875e-b1942e8fa5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:38.464136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09001e57-cf29-48dd-875e-b1942e8fa5a2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:38.464447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09001e57-cf29-48dd-875e-b1942e8fa5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:38.464726 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09001e57-cf29-48dd-875e-b1942e8fa5a2 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:38.465567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09001e57-cf29-48dd-875e-b1942e8fa5a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:38.465875 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09001e57-cf29-48dd-875e-b1942e8fa5a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:38.467399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09001e57-cf29-48dd-875e-b1942e8fa5a2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:38.479412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 116/227] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:40:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:38.480890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09001e57-cf29-48dd-875e-b1942e8fa5a2 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:38.576178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c67c9-bafa-4e55-9b4a-1508a79b258d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:38.623393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09001e57-cf29-48dd-875e-b1942e8fa5a2 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:38.625530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd3bd5-3aef-483e-91fd-1b9d76cbd31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:38.626189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd3bd5-3aef-483e-91fd-1b9d76cbd31b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:38.626473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd3bd5-3aef-483e-91fd-1b9d76cbd31b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:38.626778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd3bd5-3aef-483e-91fd-1b9d76cbd31b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:38.674846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd93222-f1ab-4877-b04e-7e571cc2d3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:38.675686 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd93222-f1ab-4877-b04e-7e571cc2d3ad None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:38.675922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd93222-f1ab-4877-b04e-7e571cc2d3ad None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:38.676696 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd93222-f1ab-4877-b04e-7e571cc2d3ad None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:38.677027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd93222-f1ab-4877-b04e-7e571cc2d3ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:38.679025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd93222-f1ab-4877-b04e-7e571cc2d3ad None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:38.706115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 117/228] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:40:38 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:40:38.707985 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd93222-f1ab-4877-b04e-7e571cc2d3ad None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:38.748691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd3bd5-3aef-483e-91fd-1b9d76cbd31b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:38.751846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd3bd5-3aef-483e-91fd-1b9d76cbd31b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:38.899483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd93222-f1ab-4877-b04e-7e571cc2d3ad None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:38.935259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 112/229] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:38.941163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd3bd5-3aef-483e-91fd-1b9d76cbd31b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:38.957826 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0684210-4b6f-4c44-b36b-7cd23cb87d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:38.959230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0684210-4b6f-4c44-b36b-7cd23cb87d65 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:38.959851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0684210-4b6f-4c44-b36b-7cd23cb87d65 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:38.960355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0684210-4b6f-4c44-b36b-7cd23cb87d65 None admin] PATH_INFO: `/v3/users/22d5df8d31334c9581ecdecb52478cac` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:38.962744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0684210-4b6f-4c44-b36b-7cd23cb87d65 None admin] RBAC: Authorizing `identity:get_user(user_id=22d5df8d31334c9581ecdecb52478cac)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:38.964457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0684210-4b6f-4c44-b36b-7cd23cb87d65 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:38.970749 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 118/230] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:38 2022] GET /identity/v3/users/22d5df8d31334c9581ecdecb52478cac => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:38.973423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0684210-4b6f-4c44-b36b-7cd23cb87d65 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:39.060530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd3bd5-3aef-483e-91fd-1b9d76cbd31b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:39.102478 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edbb41-78a2-42ce-8737-9c23287f2108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:39.103397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edbb41-78a2-42ce-8737-9c23287f2108 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:39.103676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edbb41-78a2-42ce-8737-9c23287f2108 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:39.103938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edbb41-78a2-42ce-8737-9c23287f2108 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:39.106565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edbb41-78a2-42ce-8737-9c23287f2108 None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:39.108089 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edbb41-78a2-42ce-8737-9c23287f2108 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:39.112140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 113/231] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:38 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:39.118215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edbb41-78a2-42ce-8737-9c23287f2108 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:39.139922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0684210-4b6f-4c44-b36b-7cd23cb87d65 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:39.200061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac044b-3d40-4219-aa27-985fb82f309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:39.201367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac044b-3d40-4219-aa27-985fb82f309a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:39.201851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac044b-3d40-4219-aa27-985fb82f309a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:39.202329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac044b-3d40-4219-aa27-985fb82f309a None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:39.204858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac044b-3d40-4219-aa27-985fb82f309a None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:39.207709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac044b-3d40-4219-aa27-985fb82f309a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:39.211956 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 119/232] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:38 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:39.221220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac044b-3d40-4219-aa27-985fb82f309a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:39.233854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edbb41-78a2-42ce-8737-9c23287f2108 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:39.274487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a8d4d-540c-4d2c-834e-f6d24386b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:39.275199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a8d4d-540c-4d2c-834e-f6d24386b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:39.275479 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a8d4d-540c-4d2c-834e-f6d24386b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:39.275736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a8d4d-540c-4d2c-834e-f6d24386b2b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:39.276685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a8d4d-540c-4d2c-834e-f6d24386b2b1 None admin] RBAC: Adding query filter params (role.id=90868071f00340ae853f1188777a2244, scope.project.id=e2682c48ad134e689170645b2c3b85de, user.id=9aa5e833826847db823c536f5b5e59ef) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:39.276999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a8d4d-540c-4d2c-834e-f6d24386b2b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:39.278483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a8d4d-540c-4d2c-834e-f6d24386b2b1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:39.290900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 114/233] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:39 2022] GET /identity/v3/role_assignments?user.id=9aa5e833826847db823c536f5b5e59ef&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=90868071f00340ae853f1188777a2244 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:39.361895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac044b-3d40-4219-aa27-985fb82f309a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:39.411688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d3ac-8642-417c-97bb-55015b78b406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:39.412634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d3ac-8642-417c-97bb-55015b78b406 None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:39.412910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d3ac-8642-417c-97bb-55015b78b406 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:39.413182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d3ac-8642-417c-97bb-55015b78b406 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de/users/22d5df8d31334c9581ecdecb52478cac/roles/4501501962414ed0abfba693ba95918c` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:39.418597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d3ac-8642-417c-97bb-55015b78b406 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2682c48ad134e689170645b2c3b85de, user_id=22d5df8d31334c9581ecdecb52478cac, role_id=4501501962414ed0abfba693ba95918c)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:39.420339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d3ac-8642-417c-97bb-55015b78b406 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:39.449443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 120/234] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:40:39 2022] PUT /identity/v3/projects/e2682c48ad134e689170645b2c3b85de/users/22d5df8d31334c9581ecdecb52478cac/roles/4501501962414ed0abfba693ba95918c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:40:41.464446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07379e6f-917a-4870-8b63-d032998bf957 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:41.465144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07379e6f-917a-4870-8b63-d032998bf957 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:41.465441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07379e6f-917a-4870-8b63-d032998bf957 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:41.468834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 115/235] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:41.475046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0c74c-6785-4358-be6b-708c172000a0 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:41.475807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0c74c-6785-4358-be6b-708c172000a0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:41.475807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0c74c-6785-4358-be6b-708c172000a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:41.502483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a4d0c74c-6785-4358-be6b-708c172000a0 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:41.504264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a4d0c74c-6785-4358-be6b-708c172000a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:41.617984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 121/236] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:41.665656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07379e6f-917a-4870-8b63-d032998bf957 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:41.864402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07379e6f-917a-4870-8b63-d032998bf957 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:41.906157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7e0a0-5395-4606-9d8d-861d08235ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:41.907708 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7e0a0-5395-4606-9d8d-861d08235ecb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:41.908227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7e0a0-5395-4606-9d8d-861d08235ecb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:41.908707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7e0a0-5395-4606-9d8d-861d08235ecb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:41.921941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7e0a0-5395-4606-9d8d-861d08235ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-06a7e0a0-5395-4606-9d8d-861d08235ecb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:41.933796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:41.935471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:40:41.936987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:41.940978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 116/237] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:40:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:40:41.956592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0c74c-6785-4358-be6b-708c172000a0 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:42.114786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0c74c-6785-4358-be6b-708c172000a0 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:42.159369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89a75b-43c9-43b1-ba77-9a51cbf52056 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:42.159986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89a75b-43c9-43b1-ba77-9a51cbf52056 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:42.160491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89a75b-43c9-43b1-ba77-9a51cbf52056 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:42.162452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 117/238] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:42.164731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9959db-4263-4ba3-8019-57dc5712a713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:42.165842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9959db-4263-4ba3-8019-57dc5712a713 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:42.166339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9959db-4263-4ba3-8019-57dc5712a713 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:42.166882 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9959db-4263-4ba3-8019-57dc5712a713 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:42.167953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9959db-4263-4ba3-8019-57dc5712a713 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:42.169756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a7eb3-59b1-419b-a1ae-fe778b785b8b None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:42.170250 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a7eb3-59b1-419b-a1ae-fe778b785b8b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:42.170736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a7eb3-59b1-419b-a1ae-fe778b785b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:42.171523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9959db-4263-4ba3-8019-57dc5712a713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:42.173442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9959db-4263-4ba3-8019-57dc5712a713 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:42.197566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 122/239] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:42.203625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9959db-4263-4ba3-8019-57dc5712a713 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:42.205349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f54a7eb3-59b1-419b-a1ae-fe778b785b8b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:42.207658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f54a7eb3-59b1-419b-a1ae-fe778b785b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:42.324348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 118/240] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:42.326158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6db32f-abb9-4927-879b-6f80cbf43952 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:42.326649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6db32f-abb9-4927-879b-6f80cbf43952 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:42.327146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6db32f-abb9-4927-879b-6f80cbf43952 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:42.329001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 119/241] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:42.335035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374600af-2ff5-4658-a2d2-6ae9fd7c1096 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:42.335327 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374600af-2ff5-4658-a2d2-6ae9fd7c1096 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:42.335607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374600af-2ff5-4658-a2d2-6ae9fd7c1096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:42.349475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9959db-4263-4ba3-8019-57dc5712a713 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:42.365699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-374600af-2ff5-4658-a2d2-6ae9fd7c1096 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:42.367948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-374600af-2ff5-4658-a2d2-6ae9fd7c1096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:42.406567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddea36-7da5-47d1-a7c9-335e7507255c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:42.407651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddea36-7da5-47d1-a7c9-335e7507255c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:42.407937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddea36-7da5-47d1-a7c9-335e7507255c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:42.408206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddea36-7da5-47d1-a7c9-335e7507255c None admin] PATH_INFO: `/v3/users/22d5df8d31334c9581ecdecb52478cac` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:42.409508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddea36-7da5-47d1-a7c9-335e7507255c None admin] RBAC: Authorizing `identity:get_user(user_id=22d5df8d31334c9581ecdecb52478cac)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:42.411076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddea36-7da5-47d1-a7c9-335e7507255c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:42.413543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 123/242] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:42 2022] GET /identity/v3/users/22d5df8d31334c9581ecdecb52478cac => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:42.415129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddea36-7da5-47d1-a7c9-335e7507255c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:42.470389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 120/243] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:42.471298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374600af-2ff5-4658-a2d2-6ae9fd7c1096 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:42.550474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddea36-7da5-47d1-a7c9-335e7507255c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:42.602483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95699af-5ad9-4f61-8a4d-86b08468e722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:42.603618 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95699af-5ad9-4f61-8a4d-86b08468e722 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:42.604048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95699af-5ad9-4f61-8a4d-86b08468e722 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:42.604474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95699af-5ad9-4f61-8a4d-86b08468e722 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:42.612332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374600af-2ff5-4658-a2d2-6ae9fd7c1096 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:42.616038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95699af-5ad9-4f61-8a4d-86b08468e722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-c95699af-5ad9-4f61-8a4d-86b08468e722 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:42.625778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:42.626999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:42.627993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:42.630100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 124/244] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:42.631259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95699af-5ad9-4f61-8a4d-86b08468e722 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:42.661530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c365f-e761-46aa-a56a-6f91f496072b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:42.662241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c365f-e761-46aa-a56a-6f91f496072b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:42.662560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c365f-e761-46aa-a56a-6f91f496072b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:42.662888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c365f-e761-46aa-a56a-6f91f496072b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:42.738699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c365f-e761-46aa-a56a-6f91f496072b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:42.740467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c365f-e761-46aa-a56a-6f91f496072b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:42.758616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95699af-5ad9-4f61-8a4d-86b08468e722 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:42.815928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f5582-f8e6-4e56-811b-6c39d6fb67c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:42.816764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f5582-f8e6-4e56-811b-6c39d6fb67c5 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:42.817051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f5582-f8e6-4e56-811b-6c39d6fb67c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:42.817318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f5582-f8e6-4e56-811b-6c39d6fb67c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:42.833238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f5582-f8e6-4e56-811b-6c39d6fb67c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-3e1f5582-f8e6-4e56-811b-6c39d6fb67c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:42.842937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:42.844082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:42.845038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 125/245] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:42.850131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f5582-f8e6-4e56-811b-6c39d6fb67c5 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:42.924728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 121/246] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:42.927681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec45ca2-1ba6-479f-9e39-5ec6e978379c None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:42.928172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec45ca2-1ba6-479f-9e39-5ec6e978379c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:42.928874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec45ca2-1ba6-479f-9e39-5ec6e978379c None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:42.932066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 122/247] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:42.934773 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec45ca2-1ba6-479f-9e39-5ec6e978379c None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:43.019679 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f5582-f8e6-4e56-811b-6c39d6fb67c5 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:43.077130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918888f9-ea28-41a1-8107-54a2b9c50448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:43.078199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918888f9-ea28-41a1-8107-54a2b9c50448 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.078490 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918888f9-ea28-41a1-8107-54a2b9c50448 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.079036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918888f9-ea28-41a1-8107-54a2b9c50448 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.080362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918888f9-ea28-41a1-8107-54a2b9c50448 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:43.080886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918888f9-ea28-41a1-8107-54a2b9c50448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:43.083254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918888f9-ea28-41a1-8107-54a2b9c50448 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:43.095506 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 126/248] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:43.096863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918888f9-ea28-41a1-8107-54a2b9c50448 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:43.118578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec45ca2-1ba6-479f-9e39-5ec6e978379c None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:43.167130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512aaa2-bd1e-44ef-8c65-c503f5302bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:43.167892 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512aaa2-bd1e-44ef-8c65-c503f5302bf9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.168176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512aaa2-bd1e-44ef-8c65-c503f5302bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.168585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512aaa2-bd1e-44ef-8c65-c503f5302bf9 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.169781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512aaa2-bd1e-44ef-8c65-c503f5302bf9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:43.170089 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512aaa2-bd1e-44ef-8c65-c503f5302bf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:43.171821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512aaa2-bd1e-44ef-8c65-c503f5302bf9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:43.184404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 123/249] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:43.186079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ec6e9-32ee-466c-8503-b4a0b791dd78 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.186567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ec6e9-32ee-466c-8503-b4a0b791dd78 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.187259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ec6e9-32ee-466c-8503-b4a0b791dd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.207810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-4f7ec6e9-32ee-466c-8503-b4a0b791dd78 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:43.209033 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4f7ec6e9-32ee-466c-8503-b4a0b791dd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:43.253795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918888f9-ea28-41a1-8107-54a2b9c50448 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:43.299620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a890fd3-e4de-4e12-b2d9-ec78457d0728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:43.300497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a890fd3-e4de-4e12-b2d9-ec78457d0728 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.300785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a890fd3-e4de-4e12-b2d9-ec78457d0728 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.301062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a890fd3-e4de-4e12-b2d9-ec78457d0728 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.303170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a890fd3-e4de-4e12-b2d9-ec78457d0728 None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:43.303880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a890fd3-e4de-4e12-b2d9-ec78457d0728 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:43.305750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 127/250] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:43 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:43.306953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6180688-69f4-4bb3-8664-e7952d61e0f3 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.307236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6180688-69f4-4bb3-8664-e7952d61e0f3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.307521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6180688-69f4-4bb3-8664-e7952d61e0f3 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.308969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 128/251] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:43.309526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6180688-69f4-4bb3-8664-e7952d61e0f3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:43.330737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 124/252] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:43.330737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ec6e9-32ee-466c-8503-b4a0b791dd78 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:43.446282 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6180688-69f4-4bb3-8664-e7952d61e0f3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:43.451780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ec6e9-32ee-466c-8503-b4a0b791dd78 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:43.490584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c07d3-2605-4585-b5e1-d6b483c16303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:43.492648 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c07d3-2605-4585-b5e1-d6b483c16303 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.493187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c07d3-2605-4585-b5e1-d6b483c16303 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.493637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c07d3-2605-4585-b5e1-d6b483c16303 None admin] PATH_INFO: `/v3/users/9aa5e833826847db823c536f5b5e59ef` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.495953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c07d3-2605-4585-b5e1-d6b483c16303 None admin] RBAC: Authorizing `identity:get_user(user_id=9aa5e833826847db823c536f5b5e59ef)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:43.498395 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b8999-5cf7-4606-a4a0-c5695f49db5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:43.498666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c07d3-2605-4585-b5e1-d6b483c16303 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:43.499815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b8999-5cf7-4606-a4a0-c5695f49db5c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.500246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b8999-5cf7-4606-a4a0-c5695f49db5c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.500682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b8999-5cf7-4606-a4a0-c5695f49db5c None admin] PATH_INFO: `/v3/users/c082291157e8406381c35c78363e38ce` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.501586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 125/253] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:43 2022] GET /identity/v3/users/9aa5e833826847db823c536f5b5e59ef => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:43.502979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c07d3-2605-4585-b5e1-d6b483c16303 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:43.518304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b8999-5cf7-4606-a4a0-c5695f49db5c None admin] RBAC: Authorizing `identity:get_user(user_id=c082291157e8406381c35c78363e38ce)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:43.519836 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b8999-5cf7-4606-a4a0-c5695f49db5c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:43.521750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 129/254] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:43 2022] GET /identity/v3/users/c082291157e8406381c35c78363e38ce => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:43.522997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889c21d-0a93-424c-926c-b22a398afa81 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.523811 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889c21d-0a93-424c-926c-b22a398afa81 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.524092 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889c21d-0a93-424c-926c-b22a398afa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.544004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e889c21d-0a93-424c-926c-b22a398afa81 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:43.545236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e889c21d-0a93-424c-926c-b22a398afa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:43.631016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 130/255] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:43.632325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889c21d-0a93-424c-926c-b22a398afa81 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:43.651115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c07d3-2605-4585-b5e1-d6b483c16303 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:43.694007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa5ea9-7256-4d73-9ab6-9095b71f6c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:43.694779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa5ea9-7256-4d73-9ab6-9095b71f6c53 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.695563 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa5ea9-7256-4d73-9ab6-9095b71f6c53 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.696038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa5ea9-7256-4d73-9ab6-9095b71f6c53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.696867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa5ea9-7256-4d73-9ab6-9095b71f6c53 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=e2682c48ad134e689170645b2c3b85de, user.id=22d5df8d31334c9581ecdecb52478cac) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:43.697114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa5ea9-7256-4d73-9ab6-9095b71f6c53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:43.698585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa5ea9-7256-4d73-9ab6-9095b71f6c53 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:43.705143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 126/256] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:43 2022] GET /identity/v3/role_assignments?user.id=22d5df8d31334c9581ecdecb52478cac&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:43.705982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa5ea9-7256-4d73-9ab6-9095b71f6c53 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:43.763151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889c21d-0a93-424c-926c-b22a398afa81 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:43.805250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1dd76-b89c-4afe-aa14-b53e943e5d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:43.806078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1dd76-b89c-4afe-aa14-b53e943e5d4a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.806427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1dd76-b89c-4afe-aa14-b53e943e5d4a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.806747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1dd76-b89c-4afe-aa14-b53e943e5d4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.808164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1dd76-b89c-4afe-aa14-b53e943e5d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:43.809663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1dd76-b89c-4afe-aa14-b53e943e5d4a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:43.811483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 131/257] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:43.812303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1dd76-b89c-4afe-aa14-b53e943e5d4a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:43.850978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa5ea9-7256-4d73-9ab6-9095b71f6c53 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:43.903754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918193e6-d0b4-4baa-afd1-a86b38b3eb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:43.904766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918193e6-d0b4-4baa-afd1-a86b38b3eb97 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:43.905115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918193e6-d0b4-4baa-afd1-a86b38b3eb97 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:43.905462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918193e6-d0b4-4baa-afd1-a86b38b3eb97 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:43.906970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918193e6-d0b4-4baa-afd1-a86b38b3eb97 None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:43.909334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918193e6-d0b4-4baa-afd1-a86b38b3eb97 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:43.910680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 127/258] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:43 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:43.917255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918193e6-d0b4-4baa-afd1-a86b38b3eb97 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:43.963840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1dd76-b89c-4afe-aa14-b53e943e5d4a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:44.009423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02a387-9463-47ab-ae56-11414e1d8574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:44.010547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02a387-9463-47ab-ae56-11414e1d8574 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:44.010944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02a387-9463-47ab-ae56-11414e1d8574 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:44.011219 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02a387-9463-47ab-ae56-11414e1d8574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:44.062047 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918193e6-d0b4-4baa-afd1-a86b38b3eb97 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:44.097000 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02a387-9463-47ab-ae56-11414e1d8574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:44.098481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02a387-9463-47ab-ae56-11414e1d8574 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:44.102114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fa768-9f80-4267-995d-e1b5a347b219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:44.102980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fa768-9f80-4267-995d-e1b5a347b219 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:44.103397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fa768-9f80-4267-995d-e1b5a347b219 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:44.103682 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fa768-9f80-4267-995d-e1b5a347b219 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:44.114948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fa768-9f80-4267-995d-e1b5a347b219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-379fa768-9f80-4267-995d-e1b5a347b219 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:44.123811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:44.124919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:44.125799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 128/259] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:40:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:44.126438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fa768-9f80-4267-995d-e1b5a347b219 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:44.238988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 132/260] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:44.238988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02a387-9463-47ab-ae56-11414e1d8574 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:44.290753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fa768-9f80-4267-995d-e1b5a347b219 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:44.333125 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeda240-c336-41b2-9116-8810fa83f802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:44.333804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda240-c336-41b2-9116-8810fa83f802 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:44.334156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda240-c336-41b2-9116-8810fa83f802 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:44.334458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda240-c336-41b2-9116-8810fa83f802 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:44.335113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeda240-c336-41b2-9116-8810fa83f802 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:44.336660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeda240-c336-41b2-9116-8810fa83f802 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:44.366078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 129/261] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:40:44 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:40:44.368113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeda240-c336-41b2-9116-8810fa83f802 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:44.373290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02a387-9463-47ab-ae56-11414e1d8574 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:44.441445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96bcd6-e826-46bf-92ad-a1d65ca2f10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:44.442465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96bcd6-e826-46bf-92ad-a1d65ca2f10a None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:44.442872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96bcd6-e826-46bf-92ad-a1d65ca2f10a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:44.443296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96bcd6-e826-46bf-92ad-a1d65ca2f10a None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de/users/9aa5e833826847db823c536f5b5e59ef/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:44.447081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96bcd6-e826-46bf-92ad-a1d65ca2f10a None admin] RBAC: Authorizing `identity:create_grant(project_id=e2682c48ad134e689170645b2c3b85de, user_id=9aa5e833826847db823c536f5b5e59ef, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:44.449636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96bcd6-e826-46bf-92ad-a1d65ca2f10a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:44.466843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 133/262] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:40:44 2022] PUT /identity/v3/projects/e2682c48ad134e689170645b2c3b85de/users/9aa5e833826847db823c536f5b5e59ef/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:40:44.476795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96bcd6-e826-46bf-92ad-a1d65ca2f10a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:44.531463 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeda240-c336-41b2-9116-8810fa83f802 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:44.608674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d79af-a82a-4a38-8d6f-63c22791f0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:44.610481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d79af-a82a-4a38-8d6f-63c22791f0b6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:44.611396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d79af-a82a-4a38-8d6f-63c22791f0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:44.612209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d79af-a82a-4a38-8d6f-63c22791f0b6 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:44.613896 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d79af-a82a-4a38-8d6f-63c22791f0b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:44.614838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d79af-a82a-4a38-8d6f-63c22791f0b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:44.618374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d79af-a82a-4a38-8d6f-63c22791f0b6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:44.638547 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 130/263] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:40:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:44.641059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d79af-a82a-4a38-8d6f-63c22791f0b6 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:44.664412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96bcd6-e826-46bf-92ad-a1d65ca2f10a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:44.717267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f4268-0f67-4b54-8fc7-8847fb2861ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:44.718511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f4268-0f67-4b54-8fc7-8847fb2861ea None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:44.719176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f4268-0f67-4b54-8fc7-8847fb2861ea None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:44.719797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f4268-0f67-4b54-8fc7-8847fb2861ea None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:44.721898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f4268-0f67-4b54-8fc7-8847fb2861ea None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:44.724155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f4268-0f67-4b54-8fc7-8847fb2861ea None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:44.727794 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 134/264] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:44 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:44.735490 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f4268-0f67-4b54-8fc7-8847fb2861ea None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:44.785245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d79af-a82a-4a38-8d6f-63c22791f0b6 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:44.856904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d92ea4-b340-4051-a1c1-850f8bd2dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:44.857811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d92ea4-b340-4051-a1c1-850f8bd2dc64 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:44.858147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d92ea4-b340-4051-a1c1-850f8bd2dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:44.858430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d92ea4-b340-4051-a1c1-850f8bd2dc64 None admin] PATH_INFO: `/v3/users/22d5df8d31334c9581ecdecb52478cac` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:44.861127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d92ea4-b340-4051-a1c1-850f8bd2dc64 None admin] RBAC: Authorizing `identity:get_user(user_id=22d5df8d31334c9581ecdecb52478cac)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:44.862184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d92ea4-b340-4051-a1c1-850f8bd2dc64 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:44.864119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 131/265] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:44 2022] GET /identity/v3/users/22d5df8d31334c9581ecdecb52478cac => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:44.871650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d92ea4-b340-4051-a1c1-850f8bd2dc64 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:44.877059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f4268-0f67-4b54-8fc7-8847fb2861ea None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:44.922638 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7596c49-9952-4667-bb5e-74caa97339d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:44.923638 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7596c49-9952-4667-bb5e-74caa97339d4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:44.924071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7596c49-9952-4667-bb5e-74caa97339d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:44.924533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7596c49-9952-4667-bb5e-74caa97339d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:44.925552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7596c49-9952-4667-bb5e-74caa97339d4 None admin] RBAC: Adding query filter params (role.id=a00af3df784a4cbf9d4a59157edebd16, scope.project.id=e2682c48ad134e689170645b2c3b85de, user.id=c082291157e8406381c35c78363e38ce) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:44.926066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7596c49-9952-4667-bb5e-74caa97339d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:44.928627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7596c49-9952-4667-bb5e-74caa97339d4 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:44.940721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 135/266] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:44 2022] GET /identity/v3/role_assignments?user.id=c082291157e8406381c35c78363e38ce&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=a00af3df784a4cbf9d4a59157edebd16 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:45.032078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d92ea4-b340-4051-a1c1-850f8bd2dc64 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:45.082937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d5bf8-4361-4c2a-9b30-fc1c0bd3caca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:45.083532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d5bf8-4361-4c2a-9b30-fc1c0bd3caca None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:45.083877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d5bf8-4361-4c2a-9b30-fc1c0bd3caca None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:45.084150 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d5bf8-4361-4c2a-9b30-fc1c0bd3caca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:45.201231 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d5bf8-4361-4c2a-9b30-fc1c0bd3caca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:45.203618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d5bf8-4361-4c2a-9b30-fc1c0bd3caca None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:45.349118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 132/267] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:40:45.350773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7596c49-9952-4667-bb5e-74caa97339d4 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:45.490141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7596c49-9952-4667-bb5e-74caa97339d4 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:45.535174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08506e0d-0846-4057-9861-4ed268e24563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:45.536336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08506e0d-0846-4057-9861-4ed268e24563 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:45.536815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08506e0d-0846-4057-9861-4ed268e24563 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:45.537255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08506e0d-0846-4057-9861-4ed268e24563 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:45.538614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08506e0d-0846-4057-9861-4ed268e24563 None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:45.540590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08506e0d-0846-4057-9861-4ed268e24563 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:45.543129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 136/268] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:45 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:45.551801 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d5bf8-4361-4c2a-9b30-fc1c0bd3caca None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:45.771592 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d5bf8-4361-4c2a-9b30-fc1c0bd3caca None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:45.857287 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37ecd4-a595-44aa-8a9a-37808362a881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:45.858411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37ecd4-a595-44aa-8a9a-37808362a881 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:45.858914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37ecd4-a595-44aa-8a9a-37808362a881 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:45.859386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37ecd4-a595-44aa-8a9a-37808362a881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:45.861008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37ecd4-a595-44aa-8a9a-37808362a881 None admin] RBAC: Adding query filter params (role.id=4501501962414ed0abfba693ba95918c, scope.project.id=e2682c48ad134e689170645b2c3b85de, user.id=22d5df8d31334c9581ecdecb52478cac) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:45.861564 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37ecd4-a595-44aa-8a9a-37808362a881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:45.864332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37ecd4-a595-44aa-8a9a-37808362a881 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:45.878933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 133/269] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:45 2022] GET /identity/v3/role_assignments?user.id=22d5df8d31334c9581ecdecb52478cac&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=4501501962414ed0abfba693ba95918c => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:47.635746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db954d9-2faa-4341-9c60-9518d75ef691 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:47.636874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db954d9-2faa-4341-9c60-9518d75ef691 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:47.637361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db954d9-2faa-4341-9c60-9518d75ef691 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:47.639436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 137/270] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:47.645398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cec9f3-cde0-4734-bfdd-1bcbaf1f4d48 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:47.646369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cec9f3-cde0-4734-bfdd-1bcbaf1f4d48 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:47.646928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cec9f3-cde0-4734-bfdd-1bcbaf1f4d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:47.667404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-96cec9f3-cde0-4734-bfdd-1bcbaf1f4d48 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:47.669034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-96cec9f3-cde0-4734-bfdd-1bcbaf1f4d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:47.778263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 134/271] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:47.811428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db954d9-2faa-4341-9c60-9518d75ef691 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:47.940861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464502bf-a664-4bd4-8382-864f9b842ebe None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:47.941770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464502bf-a664-4bd4-8382-864f9b842ebe None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:47.942306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464502bf-a664-4bd4-8382-864f9b842ebe None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:47.944958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 135/272] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:40:47.949610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db954d9-2faa-4341-9c60-9518d75ef691 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:47.951565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefc271-7690-47a8-89b9-5c40e3c2e61f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:47.952258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefc271-7690-47a8-89b9-5c40e3c2e61f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:47.953035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefc271-7690-47a8-89b9-5c40e3c2e61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:47.985145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-deefc271-7690-47a8-89b9-5c40e3c2e61f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:47.986921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-deefc271-7690-47a8-89b9-5c40e3c2e61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:48.003846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81bba1-e53f-46ad-8335-436a788d15f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:48.004661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81bba1-e53f-46ad-8335-436a788d15f4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:48.004937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81bba1-e53f-46ad-8335-436a788d15f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:48.005205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81bba1-e53f-46ad-8335-436a788d15f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:48.016713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81bba1-e53f-46ad-8335-436a788d15f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-4f81bba1-e53f-46ad-8335-436a788d15f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:48.025883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:48.027039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:48.028025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:48.030378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 138/273] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:48.037822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81bba1-e53f-46ad-8335-436a788d15f4 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:48.111868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 136/274] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:48.147166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefc271-7690-47a8-89b9-5c40e3c2e61f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:48.210329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81bba1-e53f-46ad-8335-436a788d15f4 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:48.259447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d6c22-ed35-45ef-9ce2-9773006f6f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:48.260904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6c22-ed35-45ef-9ce2-9773006f6f9f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:48.261553 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6c22-ed35-45ef-9ce2-9773006f6f9f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:48.261999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6c22-ed35-45ef-9ce2-9773006f6f9f None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:48.262944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d6c22-ed35-45ef-9ce2-9773006f6f9f None admin] RBAC: Adding query filter params (name=member) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:48.263414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d6c22-ed35-45ef-9ce2-9773006f6f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:48.264978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d6c22-ed35-45ef-9ce2-9773006f6f9f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:48.282406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 139/275] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:48.284039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e18dc7-22eb-4dc7-a5ed-4c6bf70476d0 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:48.284393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e18dc7-22eb-4dc7-a5ed-4c6bf70476d0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:48.284587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e18dc7-22eb-4dc7-a5ed-4c6bf70476d0 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:48.285858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 140/276] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:48.291620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e18dc7-22eb-4dc7-a5ed-4c6bf70476d0 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:48.294161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefc271-7690-47a8-89b9-5c40e3c2e61f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:48.354204 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef637bb-890f-4d65-9afc-0718779debb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:48.355034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef637bb-890f-4d65-9afc-0718779debb9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:48.355308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef637bb-890f-4d65-9afc-0718779debb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:48.355575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef637bb-890f-4d65-9afc-0718779debb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:48.356924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef637bb-890f-4d65-9afc-0718779debb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:48.358403 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef637bb-890f-4d65-9afc-0718779debb9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:48.360301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 137/277] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:48.361305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9b177-b768-4ffe-98da-e913b50f5f24 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:48.361833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9b177-b768-4ffe-98da-e913b50f5f24 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:48.362249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9b177-b768-4ffe-98da-e913b50f5f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:48.380577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-abe9b177-b768-4ffe-98da-e913b50f5f24 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:48.382165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-abe9b177-b768-4ffe-98da-e913b50f5f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:48.447358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e18dc7-22eb-4dc7-a5ed-4c6bf70476d0 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:48.473153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 138/278] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:48.475354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bfc6c-fa96-48d4-bcc4-a7df22761efb None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:48.475731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bfc6c-fa96-48d4-bcc4-a7df22761efb None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:48.476762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bfc6c-fa96-48d4-bcc4-a7df22761efb None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:48.478616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 139/279] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:48.479347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bfc6c-fa96-48d4-bcc4-a7df22761efb None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:48.502852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2eaa5d-af98-44cd-aeb8-31b598c728d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:48.504261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eaa5d-af98-44cd-aeb8-31b598c728d6 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:48.504851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eaa5d-af98-44cd-aeb8-31b598c728d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:48.505295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eaa5d-af98-44cd-aeb8-31b598c728d6 None admin] PATH_INFO: `/v3/users/9aa5e833826847db823c536f5b5e59ef` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:48.506736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2eaa5d-af98-44cd-aeb8-31b598c728d6 None admin] RBAC: Authorizing `identity:get_user(user_id=9aa5e833826847db823c536f5b5e59ef)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:48.508232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2eaa5d-af98-44cd-aeb8-31b598c728d6 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:48.509948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 141/280] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:48 2022] GET /identity/v3/users/9aa5e833826847db823c536f5b5e59ef => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:48.512947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78152ad-fc4c-4f55-aa62-ed6a4f0f87a8 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:48.513445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78152ad-fc4c-4f55-aa62-ed6a4f0f87a8 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:48.513931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78152ad-fc4c-4f55-aa62-ed6a4f0f87a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:48.540403 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b78152ad-fc4c-4f55-aa62-ed6a4f0f87a8 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:48.541947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b78152ad-fc4c-4f55-aa62-ed6a4f0f87a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:48.635778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 142/281] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:48.637871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78152ad-fc4c-4f55-aa62-ed6a4f0f87a8 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:48.659553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bfc6c-fa96-48d4-bcc4-a7df22761efb None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:48.715597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80cf1d-33ff-4efe-bdf5-d3df0d1abefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:48.716411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80cf1d-33ff-4efe-bdf5-d3df0d1abefe None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:48.716829 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80cf1d-33ff-4efe-bdf5-d3df0d1abefe None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:48.717230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80cf1d-33ff-4efe-bdf5-d3df0d1abefe None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:48.718115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80cf1d-33ff-4efe-bdf5-d3df0d1abefe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:48.719879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80cf1d-33ff-4efe-bdf5-d3df0d1abefe None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:48.758036 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 140/282] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:40:48 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:40:48.758863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80cf1d-33ff-4efe-bdf5-d3df0d1abefe None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:48.789720 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78152ad-fc4c-4f55-aa62-ed6a4f0f87a8 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:48.858885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737055f8-1241-41b0-adce-15a242f20770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:48.860285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737055f8-1241-41b0-adce-15a242f20770 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:48.860596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737055f8-1241-41b0-adce-15a242f20770 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:48.860979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737055f8-1241-41b0-adce-15a242f20770 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:48.880671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737055f8-1241-41b0-adce-15a242f20770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-737055f8-1241-41b0-adce-15a242f20770 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:48.889182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:48.890701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 143/283] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:48.892848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737055f8-1241-41b0-adce-15a242f20770 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:48.961516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80cf1d-33ff-4efe-bdf5-d3df0d1abefe None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:49.029303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f8dea-c6d5-4c2c-9db4-c3d16fa23303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:49.031346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f8dea-c6d5-4c2c-9db4-c3d16fa23303 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:49.031805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f8dea-c6d5-4c2c-9db4-c3d16fa23303 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:49.032257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f8dea-c6d5-4c2c-9db4-c3d16fa23303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:49.091740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737055f8-1241-41b0-adce-15a242f20770 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:49.155376 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9db8fa-94c6-4bc7-9442-9f550396964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:49.156262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9db8fa-94c6-4bc7-9442-9f550396964c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:49.156752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9db8fa-94c6-4bc7-9442-9f550396964c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:49.157225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9db8fa-94c6-4bc7-9442-9f550396964c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:49.158813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f8dea-c6d5-4c2c-9db4-c3d16fa23303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:49.161921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f8dea-c6d5-4c2c-9db4-c3d16fa23303 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:49.182197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9db8fa-94c6-4bc7-9442-9f550396964c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-9b9db8fa-94c6-4bc7-9442-9f550396964c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:49.197904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:49.200190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:49.202044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:49.206394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 144/284] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:49.208420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9db8fa-94c6-4bc7-9442-9f550396964c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:49.362934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 141/285] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:49.364767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f8dea-c6d5-4c2c-9db4-c3d16fa23303 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:49.390990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9db8fa-94c6-4bc7-9442-9f550396964c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:49.438523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b35ac-2215-4448-bb83-8740fdb34948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:49.439373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b35ac-2215-4448-bb83-8740fdb34948 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:49.439738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b35ac-2215-4448-bb83-8740fdb34948 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:49.440109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b35ac-2215-4448-bb83-8740fdb34948 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:49.440988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b35ac-2215-4448-bb83-8740fdb34948 None admin] RBAC: Adding query filter params (name=member) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:49.442083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b35ac-2215-4448-bb83-8740fdb34948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:49.443438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b35ac-2215-4448-bb83-8740fdb34948 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:49.456049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 145/286] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:49.457128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b35ac-2215-4448-bb83-8740fdb34948 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:49.557437 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f8dea-c6d5-4c2c-9db4-c3d16fa23303 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:49.590557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b35ac-2215-4448-bb83-8740fdb34948 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:49.621782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a2d55-4365-489c-b325-392983bfde5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:49.623331 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a2d55-4365-489c-b325-392983bfde5c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:49.624084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a2d55-4365-489c-b325-392983bfde5c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:49.624788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a2d55-4365-489c-b325-392983bfde5c None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:49.626565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a2d55-4365-489c-b325-392983bfde5c None admin] RBAC: Adding query filter params (name=reader) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:49.627593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a2d55-4365-489c-b325-392983bfde5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:49.629981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363c628-7cd5-43fe-9c32-31f3491262a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:49.631188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363c628-7cd5-43fe-9c32-31f3491262a7 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:49.631613 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a2d55-4365-489c-b325-392983bfde5c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:49.631761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363c628-7cd5-43fe-9c32-31f3491262a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:49.632020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363c628-7cd5-43fe-9c32-31f3491262a7 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:49.633991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363c628-7cd5-43fe-9c32-31f3491262a7 None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:49.635736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363c628-7cd5-43fe-9c32-31f3491262a7 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:49.638381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 146/287] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:49 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:49.639622 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363c628-7cd5-43fe-9c32-31f3491262a7 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:49.652109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 142/288] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:49.654063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a2d55-4365-489c-b325-392983bfde5c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:49.780304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363c628-7cd5-43fe-9c32-31f3491262a7 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:49.820739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc1440-f1b9-4508-b4be-bdc725e15b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:49.822136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc1440-f1b9-4508-b4be-bdc725e15b9d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:49.822588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc1440-f1b9-4508-b4be-bdc725e15b9d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:49.823086 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc1440-f1b9-4508-b4be-bdc725e15b9d None admin] PATH_INFO: `/v3/users/c8b170627cc2432c86e41a154dba78b5` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:49.836441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a2d55-4365-489c-b325-392983bfde5c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:49.841906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc1440-f1b9-4508-b4be-bdc725e15b9d None admin] RBAC: Authorizing `identity:get_user(user_id=c8b170627cc2432c86e41a154dba78b5)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:49.844283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc1440-f1b9-4508-b4be-bdc725e15b9d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:49.852053 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 147/289] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:49 2022] GET /identity/v3/users/c8b170627cc2432c86e41a154dba78b5 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:49.853144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc1440-f1b9-4508-b4be-bdc725e15b9d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:49.880754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b1dbc-54dc-4e37-8a38-da8d779cc78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:49.881785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b1dbc-54dc-4e37-8a38-da8d779cc78e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:49.882318 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b1dbc-54dc-4e37-8a38-da8d779cc78e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:49.882895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b1dbc-54dc-4e37-8a38-da8d779cc78e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:49.883947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b1dbc-54dc-4e37-8a38-da8d779cc78e None admin] RBAC: Adding query filter params (role.id=90868071f00340ae853f1188777a2244, scope.project.id=e2682c48ad134e689170645b2c3b85de, user.id=9aa5e833826847db823c536f5b5e59ef) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:49.884575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b1dbc-54dc-4e37-8a38-da8d779cc78e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:49.886410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b1dbc-54dc-4e37-8a38-da8d779cc78e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:49.893758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 143/290] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:49 2022] GET /identity/v3/role_assignments?user.id=9aa5e833826847db823c536f5b5e59ef&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=90868071f00340ae853f1188777a2244 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:49.894965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b1dbc-54dc-4e37-8a38-da8d779cc78e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:49.993455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc1440-f1b9-4508-b4be-bdc725e15b9d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:50.047888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b1dbc-54dc-4e37-8a38-da8d779cc78e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:50.058369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735769b1-5ccc-41a7-aa5c-23cbc945e9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:50.059293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735769b1-5ccc-41a7-aa5c-23cbc945e9e6 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:50.059641 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735769b1-5ccc-41a7-aa5c-23cbc945e9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:50.059963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735769b1-5ccc-41a7-aa5c-23cbc945e9e6 None admin] PATH_INFO: `/v3/users/c082291157e8406381c35c78363e38ce` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:50.061642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735769b1-5ccc-41a7-aa5c-23cbc945e9e6 None admin] RBAC: Authorizing `identity:get_user(user_id=c082291157e8406381c35c78363e38ce)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:50.064171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735769b1-5ccc-41a7-aa5c-23cbc945e9e6 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:50.067420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 148/291] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:49 2022] GET /identity/v3/users/c082291157e8406381c35c78363e38ce => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:50.075167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735769b1-5ccc-41a7-aa5c-23cbc945e9e6 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:50.118543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6590cf-e398-4e7e-9d14-a49def10786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:50.119680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6590cf-e398-4e7e-9d14-a49def10786b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:50.120002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6590cf-e398-4e7e-9d14-a49def10786b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:50.120282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6590cf-e398-4e7e-9d14-a49def10786b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:50.121409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6590cf-e398-4e7e-9d14-a49def10786b None admin] RBAC: Adding query filter params (role.id=90868071f00340ae853f1188777a2244, user.id=c8b170627cc2432c86e41a154dba78b5, scope.system=all) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:50.121776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6590cf-e398-4e7e-9d14-a49def10786b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:50.123321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6590cf-e398-4e7e-9d14-a49def10786b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:50.130424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 144/292] 10.222.0.79 () {56 vars in 1308 bytes} [Fri May 13 13:40:49 2022] GET /identity/v3/role_assignments?user.id=c8b170627cc2432c86e41a154dba78b5&scope.system=all&role.id=90868071f00340ae853f1188777a2244 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:50.275828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735769b1-5ccc-41a7-aa5c-23cbc945e9e6 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:50.337958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9726f0d-95a5-4f03-ad5c-10b2c71526ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:50.338951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9726f0d-95a5-4f03-ad5c-10b2c71526ad None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:50.339237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9726f0d-95a5-4f03-ad5c-10b2c71526ad None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:50.339537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9726f0d-95a5-4f03-ad5c-10b2c71526ad None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:50.341206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9726f0d-95a5-4f03-ad5c-10b2c71526ad None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:50.343277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9726f0d-95a5-4f03-ad5c-10b2c71526ad None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:50.350734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 149/293] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:50 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:50.353110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6590cf-e398-4e7e-9d14-a49def10786b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:50.496402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6590cf-e398-4e7e-9d14-a49def10786b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:50.543159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280d8d1-e8f6-4eda-8c6f-c047e77b2321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:50.543971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280d8d1-e8f6-4eda-8c6f-c047e77b2321 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:50.544348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280d8d1-e8f6-4eda-8c6f-c047e77b2321 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:50.544694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280d8d1-e8f6-4eda-8c6f-c047e77b2321 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de/users/c082291157e8406381c35c78363e38ce/roles/a00af3df784a4cbf9d4a59157edebd16` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:50.546923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280d8d1-e8f6-4eda-8c6f-c047e77b2321 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2682c48ad134e689170645b2c3b85de, user_id=c082291157e8406381c35c78363e38ce, role_id=a00af3df784a4cbf9d4a59157edebd16)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:50.548451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280d8d1-e8f6-4eda-8c6f-c047e77b2321 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:50.569606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 145/294] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:40:50 2022] PUT /identity/v3/projects/e2682c48ad134e689170645b2c3b85de/users/c082291157e8406381c35c78363e38ce/roles/a00af3df784a4cbf9d4a59157edebd16 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:40:52.165898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad072384-46e6-474e-8a8c-7716231f65b8 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:52.166665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad072384-46e6-474e-8a8c-7716231f65b8 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:52.166665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad072384-46e6-474e-8a8c-7716231f65b8 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:52.168602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 150/295] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:52.174630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08580f-0c56-439e-a5c3-abf9929f9a20 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:52.175375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08580f-0c56-439e-a5c3-abf9929f9a20 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:52.175375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08580f-0c56-439e-a5c3-abf9929f9a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:52.193675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-9f08580f-0c56-439e-a5c3-abf9929f9a20 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:52.195015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9f08580f-0c56-439e-a5c3-abf9929f9a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:52.274933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 146/296] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:52.294832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad072384-46e6-474e-8a8c-7716231f65b8 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:52.301292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ecab4-2648-44b4-bf35-114ff57664cf None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:52.301669 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ecab4-2648-44b4-bf35-114ff57664cf None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:52.301909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ecab4-2648-44b4-bf35-114ff57664cf None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:52.303717 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 147/297] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:40:52.309980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83609d07-55b9-4f70-8540-647653913e87 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:52.310270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83609d07-55b9-4f70-8540-647653913e87 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:52.310550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83609d07-55b9-4f70-8540-647653913e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:52.329245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-83609d07-55b9-4f70-8540-647653913e87 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:52.330591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-83609d07-55b9-4f70-8540-647653913e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:52.410837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 148/298] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:52.422191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad072384-46e6-474e-8a8c-7716231f65b8 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:52.432824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83609d07-55b9-4f70-8540-647653913e87 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:52.466124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaa0de-cb60-4cbb-8298-484708940b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:52.467417 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaa0de-cb60-4cbb-8298-484708940b4e None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:52.467696 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaa0de-cb60-4cbb-8298-484708940b4e None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:52.468069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaa0de-cb60-4cbb-8298-484708940b4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:52.469788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaa0de-cb60-4cbb-8298-484708940b4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:52.471931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaa0de-cb60-4cbb-8298-484708940b4e None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:52.474199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 151/299] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:52.481101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaa0de-cb60-4cbb-8298-484708940b4e None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:52.566015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83609d07-55b9-4f70-8540-647653913e87 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:52.599970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaa0de-cb60-4cbb-8298-484708940b4e None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:52.607374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab01bc-fa91-4a52-be01-56e648a110b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:52.608186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab01bc-fa91-4a52-be01-56e648a110b3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:52.608477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab01bc-fa91-4a52-be01-56e648a110b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:52.608757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab01bc-fa91-4a52-be01-56e648a110b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:52.621771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab01bc-fa91-4a52-be01-56e648a110b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-4fab01bc-fa91-4a52-be01-56e648a110b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:52.632028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:52.635092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:52.636913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:52.646754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 149/300] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:52.646754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab01bc-fa91-4a52-be01-56e648a110b3 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:52.653953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009cc6d-d364-4a61-b12b-01d87864aa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:52.654645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009cc6d-d364-4a61-b12b-01d87864aa39 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:52.654974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009cc6d-d364-4a61-b12b-01d87864aa39 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:52.655250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009cc6d-d364-4a61-b12b-01d87864aa39 None admin] PATH_INFO: `/v3/groups` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:52.655952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009cc6d-d364-4a61-b12b-01d87864aa39 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:52.657500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009cc6d-d364-4a61-b12b-01d87864aa39 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:52.686756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 152/301] 10.222.0.79 () {60 vars in 1134 bytes} [Fri May 13 13:40:52 2022] POST /identity/v3/groups => generated 229 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:40:52.793778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab01bc-fa91-4a52-be01-56e648a110b3 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:52.844405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e47f7-05f4-48c4-b556-cd4e17a704ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:52.845059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e47f7-05f4-48c4-b556-cd4e17a704ed None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:52.845356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e47f7-05f4-48c4-b556-cd4e17a704ed None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:52.845665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e47f7-05f4-48c4-b556-cd4e17a704ed None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:52.846448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e47f7-05f4-48c4-b556-cd4e17a704ed None admin] RBAC: Adding query filter params (name=reader) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:52.846822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e47f7-05f4-48c4-b556-cd4e17a704ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:52.848305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e47f7-05f4-48c4-b556-cd4e17a704ed None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:52.864372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 150/302] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:52.867579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009cc6d-d364-4a61-b12b-01d87864aa39 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:52.960188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8686b-3d56-4d37-8101-8379ede63bbb None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:52.960644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8686b-3d56-4d37-8101-8379ede63bbb None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:52.960935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8686b-3d56-4d37-8101-8379ede63bbb None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:52.962477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 151/303] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:40:52.968397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6384e38-8038-4301-ba0d-f3f0371fdc58 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:52.968685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6384e38-8038-4301-ba0d-f3f0371fdc58 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:52.968958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6384e38-8038-4301-ba0d-f3f0371fdc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:52.992119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f6384e38-8038-4301-ba0d-f3f0371fdc58 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:52.993577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f6384e38-8038-4301-ba0d-f3f0371fdc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:53.010574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009cc6d-d364-4a61-b12b-01d87864aa39 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:53.055682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a9069-cdb1-472e-93a9-d760ab37daeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:53.056633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a9069-cdb1-472e-93a9-d760ab37daeb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:53.057181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a9069-cdb1-472e-93a9-d760ab37daeb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:53.057466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a9069-cdb1-472e-93a9-d760ab37daeb None admin] PATH_INFO: `/v3/users/9ed55348d45644b3887442efed04a57f` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:53.076357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a9069-cdb1-472e-93a9-d760ab37daeb None admin] RBAC: Authorizing `identity:get_user(user_id=9ed55348d45644b3887442efed04a57f)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:53.078100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 152/304] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:53.079064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a9069-cdb1-472e-93a9-d760ab37daeb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:53.082319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 153/305] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:52 2022] GET /identity/v3/users/9ed55348d45644b3887442efed04a57f => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:53.089156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6384e38-8038-4301-ba0d-f3f0371fdc58 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:53.110293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a9069-cdb1-472e-93a9-d760ab37daeb None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:53.250426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a9069-cdb1-472e-93a9-d760ab37daeb None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:53.254153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6384e38-8038-4301-ba0d-f3f0371fdc58 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:53.301739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc4b97-58b6-4320-8c6a-00ecdfe3a690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:53.303372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc4b97-58b6-4320-8c6a-00ecdfe3a690 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:53.304276 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc4b97-58b6-4320-8c6a-00ecdfe3a690 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:53.304739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc4b97-58b6-4320-8c6a-00ecdfe3a690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:53.305907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc4b97-58b6-4320-8c6a-00ecdfe3a690 None admin] RBAC: Adding query filter params (role.id=a00af3df784a4cbf9d4a59157edebd16, user.id=9ed55348d45644b3887442efed04a57f, scope.system=all) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:53.306521 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc4b97-58b6-4320-8c6a-00ecdfe3a690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:53.307981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc817a-4a4e-47e4-bb5f-529602b3da99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:53.308515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc4b97-58b6-4320-8c6a-00ecdfe3a690 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:53.309203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc817a-4a4e-47e4-bb5f-529602b3da99 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:53.309779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc817a-4a4e-47e4-bb5f-529602b3da99 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:53.310458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc817a-4a4e-47e4-bb5f-529602b3da99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:53.317025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 153/306] 10.222.0.79 () {56 vars in 1308 bytes} [Fri May 13 13:40:53 2022] GET /identity/v3/role_assignments?user.id=9ed55348d45644b3887442efed04a57f&scope.system=all&role.id=a00af3df784a4cbf9d4a59157edebd16 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:53.331498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc817a-4a4e-47e4-bb5f-529602b3da99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-d6dc817a-4a4e-47e4-bb5f-529602b3da99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:53.341392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:53.342413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:53.343361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 154/307] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:40:53.351553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc4b97-58b6-4320-8c6a-00ecdfe3a690 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:53.361557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193424eb-9e51-4c86-83aa-0c5c085687f5 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:53.361736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193424eb-9e51-4c86-83aa-0c5c085687f5 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:53.362034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193424eb-9e51-4c86-83aa-0c5c085687f5 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:53.363896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 155/308] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:40:53.375412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6eae7f-2c4c-4dad-b1cf-ef09713f3a03 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:53.375788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6eae7f-2c4c-4dad-b1cf-ef09713f3a03 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:53.376324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6eae7f-2c4c-4dad-b1cf-ef09713f3a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:53.413009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-fb6eae7f-2c4c-4dad-b1cf-ef09713f3a03 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:53.415471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-fb6eae7f-2c4c-4dad-b1cf-ef09713f3a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:53.520940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc4b97-58b6-4320-8c6a-00ecdfe3a690 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:53.539401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 156/309] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:53.570581 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6eae7f-2c4c-4dad-b1cf-ef09713f3a03 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:53.593474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4431d58-e8d2-4d54-a1c0-05a289b6348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:53.595655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4431d58-e8d2-4d54-a1c0-05a289b6348b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:53.596185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4431d58-e8d2-4d54-a1c0-05a289b6348b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:53.596886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4431d58-e8d2-4d54-a1c0-05a289b6348b None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:53.598781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4431d58-e8d2-4d54-a1c0-05a289b6348b None admin] RBAC: Adding query filter params (name=member) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:53.599325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4431d58-e8d2-4d54-a1c0-05a289b6348b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:53.603429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4431d58-e8d2-4d54-a1c0-05a289b6348b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:53.626932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 154/310] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:53.634100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4431d58-e8d2-4d54-a1c0-05a289b6348b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:53.724341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6eae7f-2c4c-4dad-b1cf-ef09713f3a03 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:53.768755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359a424-ce4d-452e-b7bb-ef3cc4dc4988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:53.769662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359a424-ce4d-452e-b7bb-ef3cc4dc4988 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:53.769947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359a424-ce4d-452e-b7bb-ef3cc4dc4988 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:53.770338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359a424-ce4d-452e-b7bb-ef3cc4dc4988 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:53.793091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359a424-ce4d-452e-b7bb-ef3cc4dc4988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-3359a424-ce4d-452e-b7bb-ef3cc4dc4988 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:53.810297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:53.812180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:53.813907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:40:53.818892 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 157/311] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:40:53.820449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359a424-ce4d-452e-b7bb-ef3cc4dc4988 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:53.838086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4431d58-e8d2-4d54-a1c0-05a289b6348b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:53.891096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6b215-988d-40ff-ae52-225f28aa5c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:53.892175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6b215-988d-40ff-ae52-225f28aa5c4b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:53.892490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6b215-988d-40ff-ae52-225f28aa5c4b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:53.892771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6b215-988d-40ff-ae52-225f28aa5c4b None admin] PATH_INFO: `/v3/users/c8b170627cc2432c86e41a154dba78b5` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:53.894426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6b215-988d-40ff-ae52-225f28aa5c4b None admin] RBAC: Authorizing `identity:get_user(user_id=c8b170627cc2432c86e41a154dba78b5)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:53.895959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6b215-988d-40ff-ae52-225f28aa5c4b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:53.902866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 155/312] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:53 2022] GET /identity/v3/users/c8b170627cc2432c86e41a154dba78b5 => generated 293 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:53.907651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6b215-988d-40ff-ae52-225f28aa5c4b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:53.983014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359a424-ce4d-452e-b7bb-ef3cc4dc4988 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:54.027087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85823f8-2e2c-4183-98be-3ae1ab7f5cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:54.027772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85823f8-2e2c-4183-98be-3ae1ab7f5cfd None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:54.028060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85823f8-2e2c-4183-98be-3ae1ab7f5cfd None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:54.028336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85823f8-2e2c-4183-98be-3ae1ab7f5cfd None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:54.029095 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85823f8-2e2c-4183-98be-3ae1ab7f5cfd None admin] RBAC: Adding query filter params (name=reader) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:54.029401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85823f8-2e2c-4183-98be-3ae1ab7f5cfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:54.031062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85823f8-2e2c-4183-98be-3ae1ab7f5cfd None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:54.044728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 158/313] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:54.051135 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85823f8-2e2c-4183-98be-3ae1ab7f5cfd None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:54.087195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6b215-988d-40ff-ae52-225f28aa5c4b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:54.151784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a4dd9-0a07-4cd1-b50d-96d95458f848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:54.153125 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a4dd9-0a07-4cd1-b50d-96d95458f848 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:54.153586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a4dd9-0a07-4cd1-b50d-96d95458f848 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:54.154025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a4dd9-0a07-4cd1-b50d-96d95458f848 None admin] PATH_INFO: `/v3/system/users/c8b170627cc2432c86e41a154dba78b5/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:54.156551 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a4dd9-0a07-4cd1-b50d-96d95458f848 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c8b170627cc2432c86e41a154dba78b5, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:54.158994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a4dd9-0a07-4cd1-b50d-96d95458f848 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:54.178989 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 156/314] 10.222.0.79 () {58 vars in 1334 bytes} [Fri May 13 13:40:53 2022] PUT /identity/v3/system/users/c8b170627cc2432c86e41a154dba78b5/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:40:54.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85823f8-2e2c-4183-98be-3ae1ab7f5cfd None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:54.263192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edafe0b-0020-4313-b661-28488103e1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:54.264837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edafe0b-0020-4313-b661-28488103e1f8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:54.265320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edafe0b-0020-4313-b661-28488103e1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:54.265787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edafe0b-0020-4313-b661-28488103e1f8 None admin] PATH_INFO: `/v3/users/c082291157e8406381c35c78363e38ce` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:54.268112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edafe0b-0020-4313-b661-28488103e1f8 None admin] RBAC: Authorizing `identity:get_user(user_id=c082291157e8406381c35c78363e38ce)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:54.270924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edafe0b-0020-4313-b661-28488103e1f8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:54.275423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 159/315] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:54 2022] GET /identity/v3/users/c082291157e8406381c35c78363e38ce => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:54.284523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a4dd9-0a07-4cd1-b50d-96d95458f848 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:54.424845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a4dd9-0a07-4cd1-b50d-96d95458f848 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:54.470944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ab7a4-bbf8-4469-993d-715570affa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:54.471619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab7a4-bbf8-4469-993d-715570affa13 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:54.472006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab7a4-bbf8-4469-993d-715570affa13 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:54.472342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab7a4-bbf8-4469-993d-715570affa13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:54.557244 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ab7a4-bbf8-4469-993d-715570affa13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:54.559074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ab7a4-bbf8-4469-993d-715570affa13 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:54.679572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 157/316] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:40:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:40:54.686502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edafe0b-0020-4313-b661-28488103e1f8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:54.823169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edafe0b-0020-4313-b661-28488103e1f8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:54.876400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56ff6c-55b8-41a1-9339-7f76718c8261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:54.877725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56ff6c-55b8-41a1-9339-7f76718c8261 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:54.878150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56ff6c-55b8-41a1-9339-7f76718c8261 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:54.878565 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56ff6c-55b8-41a1-9339-7f76718c8261 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:54.880432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56ff6c-55b8-41a1-9339-7f76718c8261 None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:54.883373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56ff6c-55b8-41a1-9339-7f76718c8261 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:54.886450 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 160/317] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:40:54 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:54.894374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ab7a4-bbf8-4469-993d-715570affa13 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:55.020384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ab7a4-bbf8-4469-993d-715570affa13 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:55.063077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cec87-632a-4329-8951-289ea9f12b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:55.063757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cec87-632a-4329-8951-289ea9f12b11 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:55.064042 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cec87-632a-4329-8951-289ea9f12b11 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:55.064321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cec87-632a-4329-8951-289ea9f12b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:55.065155 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cec87-632a-4329-8951-289ea9f12b11 None admin] RBAC: Adding query filter params (role.id=a00af3df784a4cbf9d4a59157edebd16, scope.project.id=e2682c48ad134e689170645b2c3b85de, user.id=c082291157e8406381c35c78363e38ce) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:55.065469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cec87-632a-4329-8951-289ea9f12b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:55.066959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cec87-632a-4329-8951-289ea9f12b11 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:55.074720 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 158/318] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:40:54 2022] GET /identity/v3/role_assignments?user.id=c082291157e8406381c35c78363e38ce&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=a00af3df784a4cbf9d4a59157edebd16 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:55.637313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f883de-c1b4-4dce-a93b-27163a119c2c None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:55.638410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f883de-c1b4-4dce-a93b-27163a119c2c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:55.638957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f883de-c1b4-4dce-a93b-27163a119c2c None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:55.644912 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 161/319] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:55.654875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b9342-e1de-4436-93f3-f63fd9f61a7a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:55.654875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b9342-e1de-4436-93f3-f63fd9f61a7a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:55.654875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b9342-e1de-4436-93f3-f63fd9f61a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:55.674602 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-721b9342-e1de-4436-93f3-f63fd9f61a7a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:55.676014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-721b9342-e1de-4436-93f3-f63fd9f61a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:55.758145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 159/320] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:55.784462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f883de-c1b4-4dce-a93b-27163a119c2c None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:55.936664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f883de-c1b4-4dce-a93b-27163a119c2c None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:55.989846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee2dcf-b840-4cb5-99c1-f877f9c9df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:55.991053 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee2dcf-b840-4cb5-99c1-f877f9c9df67 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:55.991601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee2dcf-b840-4cb5-99c1-f877f9c9df67 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:55.992296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee2dcf-b840-4cb5-99c1-f877f9c9df67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:55.994625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee2dcf-b840-4cb5-99c1-f877f9c9df67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:55.996875 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee2dcf-b840-4cb5-99c1-f877f9c9df67 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:56.003943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 162/321] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:40:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:56.005330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee2dcf-b840-4cb5-99c1-f877f9c9df67 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:56.006245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3605b03-223c-440c-b2a0-cf1671f097ab None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:56.006786 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3605b03-223c-440c-b2a0-cf1671f097ab None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:56.007453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3605b03-223c-440c-b2a0-cf1671f097ab None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:56.009561 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 160/322] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:40:56.015059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff1832-3ec8-4f47-94a8-85a4fc1c55f4 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:56.015341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff1832-3ec8-4f47-94a8-85a4fc1c55f4 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:56.015870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff1832-3ec8-4f47-94a8-85a4fc1c55f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:56.034528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a9ff1832-3ec8-4f47-94a8-85a4fc1c55f4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:56.036115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a9ff1832-3ec8-4f47-94a8-85a4fc1c55f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:56.130736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 161/323] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:40:56.130929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee2dcf-b840-4cb5-99c1-f877f9c9df67 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:56.144272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff1832-3ec8-4f47-94a8-85a4fc1c55f4 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:56.166971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879ccc8-5924-40de-b21f-6057d8de6e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:56.167653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879ccc8-5924-40de-b21f-6057d8de6e3b None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:56.167937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879ccc8-5924-40de-b21f-6057d8de6e3b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:56.168212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879ccc8-5924-40de-b21f-6057d8de6e3b None admin] PATH_INFO: `/v3/groups` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:56.168918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879ccc8-5924-40de-b21f-6057d8de6e3b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:56.170380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879ccc8-5924-40de-b21f-6057d8de6e3b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:56.185418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 163/324] 10.222.0.79 () {60 vars in 1134 bytes} [Fri May 13 13:40:56 2022] POST /identity/v3/groups => generated 226 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:40:56.279777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff1832-3ec8-4f47-94a8-85a4fc1c55f4 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:56.319123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847e9c-05f6-44a1-92c9-16a5e247ddd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:56.320497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847e9c-05f6-44a1-92c9-16a5e247ddd5 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:56.320775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847e9c-05f6-44a1-92c9-16a5e247ddd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:56.321047 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847e9c-05f6-44a1-92c9-16a5e247ddd5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:56.333705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71847e9c-05f6-44a1-92c9-16a5e247ddd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-71847e9c-05f6-44a1-92c9-16a5e247ddd5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:56.342409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:56.343466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:56.344262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 162/325] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:40:56.350065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879ccc8-5924-40de-b21f-6057d8de6e3b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:56.492224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879ccc8-5924-40de-b21f-6057d8de6e3b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:56.535935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180fed7-2ce2-417c-b234-52fa0e289e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:56.536534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180fed7-2ce2-417c-b234-52fa0e289e17 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:56.536862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180fed7-2ce2-417c-b234-52fa0e289e17 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:56.537134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180fed7-2ce2-417c-b234-52fa0e289e17 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:56.537940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180fed7-2ce2-417c-b234-52fa0e289e17 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:56.538282 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180fed7-2ce2-417c-b234-52fa0e289e17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:56.539772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180fed7-2ce2-417c-b234-52fa0e289e17 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:56.550468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 164/326] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:56.558874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847e9c-05f6-44a1-92c9-16a5e247ddd5 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:56.695773 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847e9c-05f6-44a1-92c9-16a5e247ddd5 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:56.741966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219ca3e-b2bf-48bd-a998-f9bd01baa2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:56.743608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ca3e-b2bf-48bd-a998-f9bd01baa2c2 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:56.744092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ca3e-b2bf-48bd-a998-f9bd01baa2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:56.744584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ca3e-b2bf-48bd-a998-f9bd01baa2c2 None admin] PATH_INFO: `/v3/users/9ed55348d45644b3887442efed04a57f` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:56.746576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219ca3e-b2bf-48bd-a998-f9bd01baa2c2 None admin] RBAC: Authorizing `identity:get_user(user_id=9ed55348d45644b3887442efed04a57f)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:56.749783 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219ca3e-b2bf-48bd-a998-f9bd01baa2c2 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:56.754835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 163/327] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:56 2022] GET /identity/v3/users/9ed55348d45644b3887442efed04a57f => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:56.759102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180fed7-2ce2-417c-b234-52fa0e289e17 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:56.833165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb1ccf-de3b-4242-9aab-e5e6d60e84e3 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:56.833165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb1ccf-de3b-4242-9aab-e5e6d60e84e3 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:56.833165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb1ccf-de3b-4242-9aab-e5e6d60e84e3 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:56.834290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 164/328] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:40:56.839668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99c86f-37af-47b5-9eeb-175f40774ed8 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:56.840296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99c86f-37af-47b5-9eeb-175f40774ed8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:56.840791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99c86f-37af-47b5-9eeb-175f40774ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:56.870072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7d99c86f-37af-47b5-9eeb-175f40774ed8 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:40:56.871544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7d99c86f-37af-47b5-9eeb-175f40774ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:40:56.896360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180fed7-2ce2-417c-b234-52fa0e289e17 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:56.975567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 165/329] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:40:56.978313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0cdefd-72e3-4935-96f4-c2671ed81410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:56.979803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cdefd-72e3-4935-96f4-c2671ed81410 None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:56.980229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cdefd-72e3-4935-96f4-c2671ed81410 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:56.980644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cdefd-72e3-4935-96f4-c2671ed81410 None admin] PATH_INFO: `/v3/system/users/9ed55348d45644b3887442efed04a57f/roles/a00af3df784a4cbf9d4a59157edebd16` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:56.982780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0cdefd-72e3-4935-96f4-c2671ed81410 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9ed55348d45644b3887442efed04a57f, role_id=a00af3df784a4cbf9d4a59157edebd16)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:56.984473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0cdefd-72e3-4935-96f4-c2671ed81410 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:56.999370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 165/330] 10.222.0.79 () {58 vars in 1334 bytes} [Fri May 13 13:40:56 2022] PUT /identity/v3/system/users/9ed55348d45644b3887442efed04a57f/roles/a00af3df784a4cbf9d4a59157edebd16 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:40:57.009759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99c86f-37af-47b5-9eeb-175f40774ed8 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:57.145643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99c86f-37af-47b5-9eeb-175f40774ed8 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:57.190591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955107e-3244-40c0-895d-b79d20f8727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:57.191824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955107e-3244-40c0-895d-b79d20f8727a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:57.192330 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955107e-3244-40c0-895d-b79d20f8727a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:57.192869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955107e-3244-40c0-895d-b79d20f8727a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:57.214973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955107e-3244-40c0-895d-b79d20f8727a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-6955107e-3244-40c0-895d-b79d20f8727a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:40:57.225831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:40:57.227043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:40:57.228235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:40:57.233205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 166/331] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:40:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:40:57.240559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0cdefd-72e3-4935-96f4-c2671ed81410 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:57.430249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0cdefd-72e3-4935-96f4-c2671ed81410 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:57.486847 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42c98e-24cf-4a2f-baec-c75e0625eaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:57.487976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42c98e-24cf-4a2f-baec-c75e0625eaa1 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:57.488762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42c98e-24cf-4a2f-baec-c75e0625eaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:57.489369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42c98e-24cf-4a2f-baec-c75e0625eaa1 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:57.490360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42c98e-24cf-4a2f-baec-c75e0625eaa1 None admin] RBAC: Adding query filter params (name=member) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:57.490908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42c98e-24cf-4a2f-baec-c75e0625eaa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:57.493425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42c98e-24cf-4a2f-baec-c75e0625eaa1 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:57.508657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 166/332] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:40:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:40:57.517627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955107e-3244-40c0-895d-b79d20f8727a None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:57.705523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955107e-3244-40c0-895d-b79d20f8727a None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:57.767482 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a365d4-aabf-4b36-83ad-ac5ae7951e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:57.769469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a365d4-aabf-4b36-83ad-ac5ae7951e45 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:57.770281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a365d4-aabf-4b36-83ad-ac5ae7951e45 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:57.771153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a365d4-aabf-4b36-83ad-ac5ae7951e45 None admin] PATH_INFO: `/v3/users/c8b170627cc2432c86e41a154dba78b5` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:57.773659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a365d4-aabf-4b36-83ad-ac5ae7951e45 None admin] RBAC: Authorizing `identity:get_user(user_id=c8b170627cc2432c86e41a154dba78b5)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:57.776655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a365d4-aabf-4b36-83ad-ac5ae7951e45 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:57.779976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 167/333] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:40:57 2022] GET /identity/v3/users/c8b170627cc2432c86e41a154dba78b5 => generated 293 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:40:57.789940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42c98e-24cf-4a2f-baec-c75e0625eaa1 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:40:57.972348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42c98e-24cf-4a2f-baec-c75e0625eaa1 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:40:58.027013 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4712a-583a-4a44-8142-05c2404b4879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:40:58.027878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4712a-583a-4a44-8142-05c2404b4879 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:40:58.028224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4712a-583a-4a44-8142-05c2404b4879 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:40:58.028505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4712a-583a-4a44-8142-05c2404b4879 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:40:58.029520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4712a-583a-4a44-8142-05c2404b4879 None admin] RBAC: Adding query filter params (role.id=90868071f00340ae853f1188777a2244, user.id=c8b170627cc2432c86e41a154dba78b5, scope.system=all) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:40:58.029883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4712a-583a-4a44-8142-05c2404b4879 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:40:58.031437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4712a-583a-4a44-8142-05c2404b4879 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:40:58.049569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 167/334] 10.222.0.79 () {56 vars in 1308 bytes} [Fri May 13 13:40:57 2022] GET /identity/v3/role_assignments?user.id=c8b170627cc2432c86e41a154dba78b5&scope.system=all&role.id=90868071f00340ae853f1188777a2244 => generated 510 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:00.148203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde36d2-647f-41c5-b7da-69a20f002752 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:00.149023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde36d2-647f-41c5-b7da-69a20f002752 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:00.149153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde36d2-647f-41c5-b7da-69a20f002752 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:00.151846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 168/335] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:41:00.161074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ca762-5653-4037-a76c-40a5da05eeea None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:00.161591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ca762-5653-4037-a76c-40a5da05eeea None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:00.162083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ca762-5653-4037-a76c-40a5da05eeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:00.196790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-311ca762-5653-4037-a76c-40a5da05eeea None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:00.198874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-311ca762-5653-4037-a76c-40a5da05eeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:00.296561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 168/336] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:00.376243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde36d2-647f-41c5-b7da-69a20f002752 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:00.466854 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2606f-3efa-4778-b7d1-bf0e31d26035 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:00.467781 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2606f-3efa-4778-b7d1-bf0e31d26035 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:00.468527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2606f-3efa-4778-b7d1-bf0e31d26035 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:00.474867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 169/337] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:00.483490 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a5667-dc28-441b-be93-d76b12d7eec4 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:00.484396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a5667-dc28-441b-be93-d76b12d7eec4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:00.485155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a5667-dc28-441b-be93-d76b12d7eec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:00.531064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-887a5667-dc28-441b-be93-d76b12d7eec4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:00.532599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-887a5667-dc28-441b-be93-d76b12d7eec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:00.614100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde36d2-647f-41c5-b7da-69a20f002752 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:00.687632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a1289-88b7-4ae2-bfcb-202a5d29df9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:00.689429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a1289-88b7-4ae2-bfcb-202a5d29df9f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:00.690155 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a1289-88b7-4ae2-bfcb-202a5d29df9f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:00.690854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a1289-88b7-4ae2-bfcb-202a5d29df9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:00.693214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 170/338] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:00.695290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39bdd0-1d67-4542-89fc-9cac36603d47 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:00.695682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39bdd0-1d67-4542-89fc-9cac36603d47 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:00.696058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39bdd0-1d67-4542-89fc-9cac36603d47 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:00.701100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 171/339] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:00.707489 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0163f7-f8cf-4300-8b57-b8b1e1368af1 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:00.707892 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0163f7-f8cf-4300-8b57-b8b1e1368af1 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:00.708293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0163f7-f8cf-4300-8b57-b8b1e1368af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:00.714530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a1289-88b7-4ae2-bfcb-202a5d29df9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-133a1289-88b7-4ae2-bfcb-202a5d29df9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:00.733120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:00.735107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:00.736839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:00.739354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0c0163f7-f8cf-4300-8b57-b8b1e1368af1 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:00.740654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0c0163f7-f8cf-4300-8b57-b8b1e1368af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:00.741688 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 169/340] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:00.749049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a1289-88b7-4ae2-bfcb-202a5d29df9f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:00.891871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 172/341] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:00.893770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0163f7-f8cf-4300-8b57-b8b1e1368af1 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:00.972706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a1289-88b7-4ae2-bfcb-202a5d29df9f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:01.063959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091756e8-c7ba-43f7-8b6a-b4def81f7df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:01.065733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091756e8-c7ba-43f7-8b6a-b4def81f7df3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:01.066469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091756e8-c7ba-43f7-8b6a-b4def81f7df3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:01.067302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091756e8-c7ba-43f7-8b6a-b4def81f7df3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:01.077990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0163f7-f8cf-4300-8b57-b8b1e1368af1 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:01.085875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091756e8-c7ba-43f7-8b6a-b4def81f7df3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-091756e8-c7ba-43f7-8b6a-b4def81f7df3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:01.102951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:01.104741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:01.106361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:01.111160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 170/342] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:41:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:01.113609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d4fba-97a9-4316-8295-5c493218a65f None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:01.114377 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d4fba-97a9-4316-8295-5c493218a65f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:01.115131 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d4fba-97a9-4316-8295-5c493218a65f None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:01.118374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 171/343] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:01.120384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d4fba-97a9-4316-8295-5c493218a65f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:01.137292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d6076-67bd-4423-90c5-47079968bfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:01.138330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d6076-67bd-4423-90c5-47079968bfe4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:01.139005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d6076-67bd-4423-90c5-47079968bfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:01.139643 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d6076-67bd-4423-90c5-47079968bfe4 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:01.140908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d6076-67bd-4423-90c5-47079968bfe4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:01.141596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d6076-67bd-4423-90c5-47079968bfe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:01.144332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d6076-67bd-4423-90c5-47079968bfe4 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:01.158916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 173/344] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:41:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:01.160601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d6076-67bd-4423-90c5-47079968bfe4 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:01.296997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d4fba-97a9-4316-8295-5c493218a65f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:01.333177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d6076-67bd-4423-90c5-47079968bfe4 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:01.358086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520cf78-7ed9-409d-b81c-6c567dda2796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:01.359714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520cf78-7ed9-409d-b81c-6c567dda2796 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:01.359714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520cf78-7ed9-409d-b81c-6c567dda2796 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:01.359714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520cf78-7ed9-409d-b81c-6c567dda2796 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:01.380185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520cf78-7ed9-409d-b81c-6c567dda2796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-d520cf78-7ed9-409d-b81c-6c567dda2796 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:01.389858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:01.390911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:41:01.391845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:01.394286 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 172/345] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:01.399567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2be81-a49c-45bd-9fe8-872082734fc1 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:01.400094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2be81-a49c-45bd-9fe8-872082734fc1 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:01.400541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2be81-a49c-45bd-9fe8-872082734fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:01.402321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d2bbb-34ea-4e9d-a3e3-e567e8088d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:01.403601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d2bbb-34ea-4e9d-a3e3-e567e8088d82 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:01.404046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d2bbb-34ea-4e9d-a3e3-e567e8088d82 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:01.404567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d2bbb-34ea-4e9d-a3e3-e567e8088d82 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:01.406276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d2bbb-34ea-4e9d-a3e3-e567e8088d82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:01.408145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d2bbb-34ea-4e9d-a3e3-e567e8088d82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:01.411829 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d2bbb-34ea-4e9d-a3e3-e567e8088d82 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:01.422342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 174/346] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:01.423698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d2bbb-34ea-4e9d-a3e3-e567e8088d82 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:01.424552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-0ec2be81-a49c-45bd-9fe8-872082734fc1 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:01.425756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0ec2be81-a49c-45bd-9fe8-872082734fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:01.535385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 173/347] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:01.536521 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e732ad-1db4-4a95-a52b-d63863d8a0e3 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:01.537054 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e732ad-1db4-4a95-a52b-d63863d8a0e3 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:01.537512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e732ad-1db4-4a95-a52b-d63863d8a0e3 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:01.539605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 174/348] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:01.540874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e732ad-1db4-4a95-a52b-d63863d8a0e3 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:01.583108 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d2bbb-34ea-4e9d-a3e3-e567e8088d82 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:01.632119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae860a3d-bca1-4508-9f62-5bbfa8c807cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:01.633049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae860a3d-bca1-4508-9f62-5bbfa8c807cb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:01.633502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae860a3d-bca1-4508-9f62-5bbfa8c807cb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:01.633772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae860a3d-bca1-4508-9f62-5bbfa8c807cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:01.692269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e732ad-1db4-4a95-a52b-d63863d8a0e3 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:01.740795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae860a3d-bca1-4508-9f62-5bbfa8c807cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:01.743405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae860a3d-bca1-4508-9f62-5bbfa8c807cb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:01.756999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11aed4d-4192-40e9-9335-6da475903ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:01.758011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11aed4d-4192-40e9-9335-6da475903ee9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:01.758447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11aed4d-4192-40e9-9335-6da475903ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:01.758947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11aed4d-4192-40e9-9335-6da475903ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:01.759895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11aed4d-4192-40e9-9335-6da475903ee9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:01.760193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11aed4d-4192-40e9-9335-6da475903ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:01.761784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11aed4d-4192-40e9-9335-6da475903ee9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:01.771861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 175/349] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:41:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:01.772788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11aed4d-4192-40e9-9335-6da475903ee9 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:01.901658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 175/350] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:01.903501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65320139-eb9f-49b8-88d4-370ee07feba9 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:01.904040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65320139-eb9f-49b8-88d4-370ee07feba9 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:01.904727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65320139-eb9f-49b8-88d4-370ee07feba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:01.931692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-65320139-eb9f-49b8-88d4-370ee07feba9 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:01.934018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-65320139-eb9f-49b8-88d4-370ee07feba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:01.957903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11aed4d-4192-40e9-9335-6da475903ee9 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:02.014591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcd71f-0137-4694-8e4a-e6df2a390278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:02.014591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcd71f-0137-4694-8e4a-e6df2a390278 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:02.014591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcd71f-0137-4694-8e4a-e6df2a390278 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:02.014591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcd71f-0137-4694-8e4a-e6df2a390278 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:02.046414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 176/351] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:02.047662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65320139-eb9f-49b8-88d4-370ee07feba9 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:02.105052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcd71f-0137-4694-8e4a-e6df2a390278 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:02.106525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcd71f-0137-4694-8e4a-e6df2a390278 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:02.166361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65320139-eb9f-49b8-88d4-370ee07feba9 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:02.209383 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22469b-809e-4bd4-8e37-a667aaa11383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:02.210188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22469b-809e-4bd4-8e37-a667aaa11383 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:02.210822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22469b-809e-4bd4-8e37-a667aaa11383 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:02.210822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22469b-809e-4bd4-8e37-a667aaa11383 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:02.220818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 176/352] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:02.221929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcd71f-0137-4694-8e4a-e6df2a390278 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:02.225343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22469b-809e-4bd4-8e37-a667aaa11383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-ac22469b-809e-4bd4-8e37-a667aaa11383 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:02.234924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:02.235889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:02.236668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 177/353] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:41:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:02.237333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22469b-809e-4bd4-8e37-a667aaa11383 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:02.331784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcd71f-0137-4694-8e4a-e6df2a390278 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:02.362412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22469b-809e-4bd4-8e37-a667aaa11383 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:02.371266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d722a60-46d8-4397-b95d-e7fb33947f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:02.372153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d722a60-46d8-4397-b95d-e7fb33947f94 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:02.372430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d722a60-46d8-4397-b95d-e7fb33947f94 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:02.372702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d722a60-46d8-4397-b95d-e7fb33947f94 None admin] PATH_INFO: `/v3/users/9ed55348d45644b3887442efed04a57f` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:02.373978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d722a60-46d8-4397-b95d-e7fb33947f94 None admin] RBAC: Authorizing `identity:get_user(user_id=9ed55348d45644b3887442efed04a57f)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:02.375601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d722a60-46d8-4397-b95d-e7fb33947f94 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:02.378516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 177/354] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:41:02 2022] GET /identity/v3/users/9ed55348d45644b3887442efed04a57f => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:02.378516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d722a60-46d8-4397-b95d-e7fb33947f94 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:02.407473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16573fc-98ee-4687-9e99-1b71075d2f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:02.408378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16573fc-98ee-4687-9e99-1b71075d2f59 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:02.408621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16573fc-98ee-4687-9e99-1b71075d2f59 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:02.408893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16573fc-98ee-4687-9e99-1b71075d2f59 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:02.410557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16573fc-98ee-4687-9e99-1b71075d2f59 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:02.412047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16573fc-98ee-4687-9e99-1b71075d2f59 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:02.414531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 178/355] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:02 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:02.415822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7cea0-3682-415d-b4d5-ebc3035c473b None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:02.416109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7cea0-3682-415d-b4d5-ebc3035c473b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:02.416377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7cea0-3682-415d-b4d5-ebc3035c473b None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:02.417876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 179/356] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:02.418480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7cea0-3682-415d-b4d5-ebc3035c473b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:02.504416 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d722a60-46d8-4397-b95d-e7fb33947f94 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:02.555161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd3241-2384-4350-9b08-4e7f95304c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:02.556442 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd3241-2384-4350-9b08-4e7f95304c49 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:02.556911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd3241-2384-4350-9b08-4e7f95304c49 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:02.557369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd3241-2384-4350-9b08-4e7f95304c49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:02.567160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7cea0-3682-415d-b4d5-ebc3035c473b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:02.573229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd3241-2384-4350-9b08-4e7f95304c49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-7bdd3241-2384-4350-9b08-4e7f95304c49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:02.582580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:02.584050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 178/357] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:02.585336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd3241-2384-4350-9b08-4e7f95304c49 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:02.617300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58db1f-8780-4d9b-acc8-aa380d0a6c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:02.618075 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58db1f-8780-4d9b-acc8-aa380d0a6c2d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:02.618346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58db1f-8780-4d9b-acc8-aa380d0a6c2d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:02.618659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58db1f-8780-4d9b-acc8-aa380d0a6c2d None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:02.620015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58db1f-8780-4d9b-acc8-aa380d0a6c2d None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:02.621487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58db1f-8780-4d9b-acc8-aa380d0a6c2d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:02.623614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 180/358] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:02 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:02.624458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58db1f-8780-4d9b-acc8-aa380d0a6c2d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:02.733148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd3241-2384-4350-9b08-4e7f95304c49 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:02.753404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58db1f-8780-4d9b-acc8-aa380d0a6c2d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:02.780236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ef712-ed21-4020-9e72-af097115179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:02.780884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ef712-ed21-4020-9e72-af097115179b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:02.781227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ef712-ed21-4020-9e72-af097115179b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:02.781509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ef712-ed21-4020-9e72-af097115179b None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:02.782562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ef712-ed21-4020-9e72-af097115179b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:02.782562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ef712-ed21-4020-9e72-af097115179b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:02.784027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ef712-ed21-4020-9e72-af097115179b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:02.794769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 179/359] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:02.796550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc1573-3366-4a92-81ad-484f9937f247 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:02.797014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc1573-3366-4a92-81ad-484f9937f247 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:02.797447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc1573-3366-4a92-81ad-484f9937f247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:02.804969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35158fdf-d154-402a-b148-73e707ccd9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:02.806169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35158fdf-d154-402a-b148-73e707ccd9e1 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:02.806169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35158fdf-d154-402a-b148-73e707ccd9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:02.806368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35158fdf-d154-402a-b148-73e707ccd9e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:02.809086 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35158fdf-d154-402a-b148-73e707ccd9e1 None admin] RBAC: Adding query filter params (role.id=a00af3df784a4cbf9d4a59157edebd16, user.id=9ed55348d45644b3887442efed04a57f, scope.system=all) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:02.809086 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35158fdf-d154-402a-b148-73e707ccd9e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:02.809276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35158fdf-d154-402a-b148-73e707ccd9e1 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:02.815187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-d6bc1573-3366-4a92-81ad-484f9937f247 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:02.816509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d6bc1573-3366-4a92-81ad-484f9937f247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:02.824180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 181/360] 10.222.0.79 () {56 vars in 1308 bytes} [Fri May 13 13:41:02 2022] GET /identity/v3/role_assignments?user.id=9ed55348d45644b3887442efed04a57f&scope.system=all&role.id=a00af3df784a4cbf9d4a59157edebd16 => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:02.824883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35158fdf-d154-402a-b148-73e707ccd9e1 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:02.937574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 180/361] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:02.938518 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc1573-3366-4a92-81ad-484f9937f247 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:02.983394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35158fdf-d154-402a-b148-73e707ccd9e1 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:03.030202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20c80c-f9d0-4fd1-926a-6d5ec2ae8f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:03.031646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20c80c-f9d0-4fd1-926a-6d5ec2ae8f5c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:03.031646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20c80c-f9d0-4fd1-926a-6d5ec2ae8f5c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:03.031840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20c80c-f9d0-4fd1-926a-6d5ec2ae8f5c None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:03.039787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20c80c-f9d0-4fd1-926a-6d5ec2ae8f5c None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:03.041443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20c80c-f9d0-4fd1-926a-6d5ec2ae8f5c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:03.043650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 182/362] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:02 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:03.044798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20c80c-f9d0-4fd1-926a-6d5ec2ae8f5c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:03.099142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc1573-3366-4a92-81ad-484f9937f247 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:03.150654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37b38d-ec16-4ac7-a3ca-234aac39133e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:03.152079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37b38d-ec16-4ac7-a3ca-234aac39133e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:03.152079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37b38d-ec16-4ac7-a3ca-234aac39133e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:03.152079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37b38d-ec16-4ac7-a3ca-234aac39133e None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:03.152701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37b38d-ec16-4ac7-a3ca-234aac39133e None admin] RBAC: Adding query filter params (name=member) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:03.152944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37b38d-ec16-4ac7-a3ca-234aac39133e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:03.154366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37b38d-ec16-4ac7-a3ca-234aac39133e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:03.167552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 181/363] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:41:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:03.169111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37b38d-ec16-4ac7-a3ca-234aac39133e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:03.192748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20c80c-f9d0-4fd1-926a-6d5ec2ae8f5c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:03.242140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4667f3-134d-4e39-82b9-83e83a37b752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:03.243166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4667f3-134d-4e39-82b9-83e83a37b752 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:03.243445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4667f3-134d-4e39-82b9-83e83a37b752 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:03.243716 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4667f3-134d-4e39-82b9-83e83a37b752 None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:03.246102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4667f3-134d-4e39-82b9-83e83a37b752 None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:03.248933 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4667f3-134d-4e39-82b9-83e83a37b752 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:03.252171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 183/364] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:03 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:03.253766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4667f3-134d-4e39-82b9-83e83a37b752 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:03.287248 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37b38d-ec16-4ac7-a3ca-234aac39133e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:03.336233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba13ce8-a298-4fe8-b243-f7114407f565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:03.337262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba13ce8-a298-4fe8-b243-f7114407f565 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:03.338230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba13ce8-a298-4fe8-b243-f7114407f565 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:03.338684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba13ce8-a298-4fe8-b243-f7114407f565 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:03.414121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4667f3-134d-4e39-82b9-83e83a37b752 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:03.433729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba13ce8-a298-4fe8-b243-f7114407f565 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:03.436097 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba13ce8-a298-4fe8-b243-f7114407f565 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:03.464391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096183d7-c867-4283-9cc3-a2e2cbbd4da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:03.465633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096183d7-c867-4283-9cc3-a2e2cbbd4da9 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:03.466088 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096183d7-c867-4283-9cc3-a2e2cbbd4da9 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:03.466532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096183d7-c867-4283-9cc3-a2e2cbbd4da9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:03.482036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096183d7-c867-4283-9cc3-a2e2cbbd4da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-096183d7-c867-4283-9cc3-a2e2cbbd4da9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:03.496164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:03.497344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 184/365] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:41:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:03.503028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096183d7-c867-4283-9cc3-a2e2cbbd4da9 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:03.574600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 182/366] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:03.575421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba13ce8-a298-4fe8-b243-f7114407f565 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:03.617020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096183d7-c867-4283-9cc3-a2e2cbbd4da9 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:03.652272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3fc1d-c344-4cfe-bbea-77c9f3ce15c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:03.652926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3fc1d-c344-4cfe-bbea-77c9f3ce15c6 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:03.653205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3fc1d-c344-4cfe-bbea-77c9f3ce15c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:03.653479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3fc1d-c344-4cfe-bbea-77c9f3ce15c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:03.654310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3fc1d-c344-4cfe-bbea-77c9f3ce15c6 None admin] RBAC: Adding query filter params (group.id=a7fc3dcf59e84e65b6807bebfd0f9987, role.id=90868071f00340ae853f1188777a2244, scope.project.id=8ff5716d18b14f8c83813becdf6695bd) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:03.654625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3fc1d-c344-4cfe-bbea-77c9f3ce15c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:03.656112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3fc1d-c344-4cfe-bbea-77c9f3ce15c6 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:03.661852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 185/367] 10.222.0.79 () {56 vars in 1376 bytes} [Fri May 13 13:41:03 2022] GET /identity/v3/role_assignments?group.id=a7fc3dcf59e84e65b6807bebfd0f9987&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=90868071f00340ae853f1188777a2244 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:03.662662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3fc1d-c344-4cfe-bbea-77c9f3ce15c6 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:03.735092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba13ce8-a298-4fe8-b243-f7114407f565 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:03.780044 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6893153-a805-483b-988c-f923ceaa0d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:03.780719 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6893153-a805-483b-988c-f923ceaa0d96 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:03.781001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6893153-a805-483b-988c-f923ceaa0d96 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:03.781278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6893153-a805-483b-988c-f923ceaa0d96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:03.807973 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3fc1d-c344-4cfe-bbea-77c9f3ce15c6 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:03.853062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573930b5-0b8d-4320-ba19-e7482ff85542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:03.853961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573930b5-0b8d-4320-ba19-e7482ff85542 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:03.854365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573930b5-0b8d-4320-ba19-e7482ff85542 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:03.854817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573930b5-0b8d-4320-ba19-e7482ff85542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:03.855987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573930b5-0b8d-4320-ba19-e7482ff85542 None admin] RBAC: Adding query filter params (group.id=a7fc3dcf59e84e65b6807bebfd0f9987, role.id=4501501962414ed0abfba693ba95918c, scope.project.id=e2682c48ad134e689170645b2c3b85de) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:03.856497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573930b5-0b8d-4320-ba19-e7482ff85542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:03.858416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573930b5-0b8d-4320-ba19-e7482ff85542 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:03.866151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 186/368] 10.222.0.79 () {56 vars in 1376 bytes} [Fri May 13 13:41:03 2022] GET /identity/v3/role_assignments?group.id=a7fc3dcf59e84e65b6807bebfd0f9987&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=4501501962414ed0abfba693ba95918c => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:03.867800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573930b5-0b8d-4320-ba19-e7482ff85542 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:03.869268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6893153-a805-483b-988c-f923ceaa0d96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:03.872326 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6893153-a805-483b-988c-f923ceaa0d96 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:04.021997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 183/369] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:04.022964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6893153-a805-483b-988c-f923ceaa0d96 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:04.027966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573930b5-0b8d-4320-ba19-e7482ff85542 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:04.070281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ed4c0-aa1c-482c-b018-9b1f767b4e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:04.071370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ed4c0-aa1c-482c-b018-9b1f767b4e4b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:04.071862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ed4c0-aa1c-482c-b018-9b1f767b4e4b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:04.072357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ed4c0-aa1c-482c-b018-9b1f767b4e4b None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:04.073495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ed4c0-aa1c-482c-b018-9b1f767b4e4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:04.074165 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ed4c0-aa1c-482c-b018-9b1f767b4e4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:04.076266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ed4c0-aa1c-482c-b018-9b1f767b4e4b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:04.087076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 187/370] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:04.088272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ed4c0-aa1c-482c-b018-9b1f767b4e4b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:04.146108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6893153-a805-483b-988c-f923ceaa0d96 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:04.185264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c4ef3-273f-4e22-ab15-fda1d99fbaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:04.186175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c4ef3-273f-4e22-ab15-fda1d99fbaa3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:04.186452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c4ef3-273f-4e22-ab15-fda1d99fbaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:04.186737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c4ef3-273f-4e22-ab15-fda1d99fbaa3 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:04.188065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c4ef3-273f-4e22-ab15-fda1d99fbaa3 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:04.190905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c4ef3-273f-4e22-ab15-fda1d99fbaa3 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:04.192752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 184/371] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:04 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:04.193586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c4ef3-273f-4e22-ab15-fda1d99fbaa3 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:04.228227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ed4c0-aa1c-482c-b018-9b1f767b4e4b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:04.271943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efc0a0-a96a-4a18-aa56-47ebec3c7667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:04.274406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efc0a0-a96a-4a18-aa56-47ebec3c7667 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:04.275189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efc0a0-a96a-4a18-aa56-47ebec3c7667 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:04.275950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efc0a0-a96a-4a18-aa56-47ebec3c7667 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:04.278494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efc0a0-a96a-4a18-aa56-47ebec3c7667 None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:04.281578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efc0a0-a96a-4a18-aa56-47ebec3c7667 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:04.285530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 188/372] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:04 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:04.287504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efc0a0-a96a-4a18-aa56-47ebec3c7667 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:04.308727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c4ef3-273f-4e22-ab15-fda1d99fbaa3 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:04.345792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f34aac-c33e-4e20-bad7-13582faf1f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:04.346485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f34aac-c33e-4e20-bad7-13582faf1f10 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:04.346732 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f34aac-c33e-4e20-bad7-13582faf1f10 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:04.347016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f34aac-c33e-4e20-bad7-13582faf1f10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:04.418089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efc0a0-a96a-4a18-aa56-47ebec3c7667 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:04.428723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f34aac-c33e-4e20-bad7-13582faf1f10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:04.431130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f34aac-c33e-4e20-bad7-13582faf1f10 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:04.462473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33737c8-69e1-454e-a750-e554ee68f810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:04.463763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33737c8-69e1-454e-a750-e554ee68f810 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:04.464297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33737c8-69e1-454e-a750-e554ee68f810 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:04.464794 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33737c8-69e1-454e-a750-e554ee68f810 None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:04.466330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33737c8-69e1-454e-a750-e554ee68f810 None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:04.468061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33737c8-69e1-454e-a750-e554ee68f810 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:04.470013 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 189/373] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:04 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:04.471052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33737c8-69e1-454e-a750-e554ee68f810 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:04.560588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 185/374] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:04.561583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f34aac-c33e-4e20-bad7-13582faf1f10 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:04.581190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33737c8-69e1-454e-a750-e554ee68f810 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:04.617912 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c7bb3-0221-4f34-ad9f-d4d875d9fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:04.619207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7bb3-0221-4f34-ad9f-d4d875d9fdbf None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:04.619721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7bb3-0221-4f34-ad9f-d4d875d9fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:04.620209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7bb3-0221-4f34-ad9f-d4d875d9fdbf None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:04.621675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7bb3-0221-4f34-ad9f-d4d875d9fdbf None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:04.623420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7bb3-0221-4f34-ad9f-d4d875d9fdbf None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:04.625643 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 190/375] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:04 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:04.626672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c7bb3-0221-4f34-ad9f-d4d875d9fdbf None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:04.678060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f34aac-c33e-4e20-bad7-13582faf1f10 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:04.714448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b451d5-f7d7-4ed8-ad23-f845d7d8af96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:04.715075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b451d5-f7d7-4ed8-ad23-f845d7d8af96 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:04.715350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b451d5-f7d7-4ed8-ad23-f845d7d8af96 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:04.715618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b451d5-f7d7-4ed8-ad23-f845d7d8af96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:04.716437 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b451d5-f7d7-4ed8-ad23-f845d7d8af96 None admin] RBAC: Adding query filter params (group.id=a7fc3dcf59e84e65b6807bebfd0f9987, role.id=4501501962414ed0abfba693ba95918c, scope.project.id=8ff5716d18b14f8c83813becdf6695bd) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:04.716813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b451d5-f7d7-4ed8-ad23-f845d7d8af96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:04.718301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b451d5-f7d7-4ed8-ad23-f845d7d8af96 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:04.724511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 186/376] 10.222.0.79 () {56 vars in 1376 bytes} [Fri May 13 13:41:04 2022] GET /identity/v3/role_assignments?group.id=a7fc3dcf59e84e65b6807bebfd0f9987&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=4501501962414ed0abfba693ba95918c => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:04.725489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b451d5-f7d7-4ed8-ad23-f845d7d8af96 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:04.736062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c7bb3-0221-4f34-ad9f-d4d875d9fdbf None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:04.783664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f87910-eb57-47e1-9a20-6b63a5d55869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:04.785009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f87910-eb57-47e1-9a20-6b63a5d55869 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:04.785832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f87910-eb57-47e1-9a20-6b63a5d55869 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:04.786529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f87910-eb57-47e1-9a20-6b63a5d55869 None admin] PATH_INFO: `/v3/projects/584c523555e7486e977f27ddcc51047b` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:04.788324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f87910-eb57-47e1-9a20-6b63a5d55869 None admin] RBAC: Authorizing `identity:get_project(project_id=584c523555e7486e977f27ddcc51047b)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:04.790243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f87910-eb57-47e1-9a20-6b63a5d55869 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:04.792804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 191/377] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:04 2022] GET /identity/v3/projects/584c523555e7486e977f27ddcc51047b => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:04.800846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f87910-eb57-47e1-9a20-6b63a5d55869 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:04.849467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b451d5-f7d7-4ed8-ad23-f845d7d8af96 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:04.889432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6c892-f85b-4c59-a6e7-451d6310f91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:04.890112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6c892-f85b-4c59-a6e7-451d6310f91c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:04.890387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6c892-f85b-4c59-a6e7-451d6310f91c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:04.890658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6c892-f85b-4c59-a6e7-451d6310f91c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:04.891529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6c892-f85b-4c59-a6e7-451d6310f91c None admin] RBAC: Adding query filter params (group.id=a7fc3dcf59e84e65b6807bebfd0f9987, role.id=90868071f00340ae853f1188777a2244, scope.project.id=e2682c48ad134e689170645b2c3b85de) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:04.891838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6c892-f85b-4c59-a6e7-451d6310f91c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:04.893321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6c892-f85b-4c59-a6e7-451d6310f91c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:04.900946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 187/378] 10.222.0.79 () {56 vars in 1376 bytes} [Fri May 13 13:41:04 2022] GET /identity/v3/role_assignments?group.id=a7fc3dcf59e84e65b6807bebfd0f9987&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=90868071f00340ae853f1188777a2244 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:05.017448 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f87910-eb57-47e1-9a20-6b63a5d55869 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:05.078109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674483a1-70ec-4eab-a65d-8d42da0f77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:05.079056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674483a1-70ec-4eab-a65d-8d42da0f77ac None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:05.079337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674483a1-70ec-4eab-a65d-8d42da0f77ac None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:05.079609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674483a1-70ec-4eab-a65d-8d42da0f77ac None admin] PATH_INFO: `/v3/groups/309505a759b04402a153d6d1086de51a` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:05.090925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674483a1-70ec-4eab-a65d-8d42da0f77ac None admin] RBAC: Authorizing `identity:get_group(group_id=309505a759b04402a153d6d1086de51a)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:05.092453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674483a1-70ec-4eab-a65d-8d42da0f77ac None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:05.094401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 192/379] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:04 2022] GET /identity/v3/groups/309505a759b04402a153d6d1086de51a => generated 229 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:05.101412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6c892-f85b-4c59-a6e7-451d6310f91c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:05.245029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6c892-f85b-4c59-a6e7-451d6310f91c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:05.283700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2264d-6c97-446e-8d98-4bb741554e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:05.284451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2264d-6c97-446e-8d98-4bb741554e93 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:05.284794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2264d-6c97-446e-8d98-4bb741554e93 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:05.285148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2264d-6c97-446e-8d98-4bb741554e93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:05.286094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2264d-6c97-446e-8d98-4bb741554e93 None admin] RBAC: Adding query filter params (group.id=309505a759b04402a153d6d1086de51a, role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=584c523555e7486e977f27ddcc51047b) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:05.286400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2264d-6c97-446e-8d98-4bb741554e93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:05.289002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2264d-6c97-446e-8d98-4bb741554e93 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:05.296490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 188/380] 10.222.0.79 () {56 vars in 1376 bytes} [Fri May 13 13:41:05 2022] GET /identity/v3/role_assignments?group.id=309505a759b04402a153d6d1086de51a&scope.project.id=584c523555e7486e977f27ddcc51047b&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:06.550429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0562e-5582-497d-a2e2-eab0a70284ed None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:06.551497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0562e-5582-497d-a2e2-eab0a70284ed None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:06.551991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0562e-5582-497d-a2e2-eab0a70284ed None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:06.553779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 193/381] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:06.559287 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8647b-16da-4de3-9ff7-f73b1f2c43b3 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:06.559763 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8647b-16da-4de3-9ff7-f73b1f2c43b3 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:06.559847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8647b-16da-4de3-9ff7-f73b1f2c43b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:06.582214 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b5a8647b-16da-4de3-9ff7-f73b1f2c43b3 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:06.584017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b5a8647b-16da-4de3-9ff7-f73b1f2c43b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:06.659337 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 189/382] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:06.682122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0562e-5582-497d-a2e2-eab0a70284ed None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:06.826485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0562e-5582-497d-a2e2-eab0a70284ed None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:06.901271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d054bb-5595-40d4-9426-01e14b010815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:06.902568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d054bb-5595-40d4-9426-01e14b010815 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:06.903071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d054bb-5595-40d4-9426-01e14b010815 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:06.903535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d054bb-5595-40d4-9426-01e14b010815 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:06.932722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d054bb-5595-40d4-9426-01e14b010815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-27d054bb-5595-40d4-9426-01e14b010815 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:06.945904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:06.947039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:06.947819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:06.956713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 194/383] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:06.963044 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8647b-16da-4de3-9ff7-f73b1f2c43b3 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:07.128858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8647b-16da-4de3-9ff7-f73b1f2c43b3 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:07.184161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e89f0-0b37-4413-9765-d7ce786757bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:07.184858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e89f0-0b37-4413-9765-d7ce786757bd None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:07.185144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e89f0-0b37-4413-9765-d7ce786757bd None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:07.185420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e89f0-0b37-4413-9765-d7ce786757bd None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:07.186172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e89f0-0b37-4413-9765-d7ce786757bd None admin] RBAC: Adding query filter params (name=member) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:07.186474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e89f0-0b37-4413-9765-d7ce786757bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:07.187924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e89f0-0b37-4413-9765-d7ce786757bd None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:07.202825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 190/384] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:41:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:07.211665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d054bb-5595-40d4-9426-01e14b010815 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:07.343462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d054bb-5595-40d4-9426-01e14b010815 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:07.396352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b365aa-d609-4c81-aa75-9a3eb213dd83 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:07.396974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b365aa-d609-4c81-aa75-9a3eb213dd83 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:07.397492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b365aa-d609-4c81-aa75-9a3eb213dd83 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:07.399354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 191/385] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:41:07.402741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32370ca8-54ca-4b0c-9503-e6e6996d9b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:07.402741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32370ca8-54ca-4b0c-9503-e6e6996d9b1d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:07.402741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32370ca8-54ca-4b0c-9503-e6e6996d9b1d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:07.402741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32370ca8-54ca-4b0c-9503-e6e6996d9b1d None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:07.404312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32370ca8-54ca-4b0c-9503-e6e6996d9b1d None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:07.405430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36dfd-11a8-4284-be18-70e574ee212e None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:07.405748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32370ca8-54ca-4b0c-9503-e6e6996d9b1d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:07.406129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36dfd-11a8-4284-be18-70e574ee212e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:07.406611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36dfd-11a8-4284-be18-70e574ee212e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:07.407681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 195/386] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:07 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:07.414243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32370ca8-54ca-4b0c-9503-e6e6996d9b1d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:07.439915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-fdc36dfd-11a8-4284-be18-70e574ee212e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:07.441217 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-fdc36dfd-11a8-4284-be18-70e574ee212e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:07.596121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 192/387] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:07.607652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32370ca8-54ca-4b0c-9503-e6e6996d9b1d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:07.625429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc36dfd-11a8-4284-be18-70e574ee212e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:07.684045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca762ec-c5de-4dd5-9d3f-0c2d3f2cb88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:07.684976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca762ec-c5de-4dd5-9d3f-0c2d3f2cb88b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:07.685261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca762ec-c5de-4dd5-9d3f-0c2d3f2cb88b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:07.685540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca762ec-c5de-4dd5-9d3f-0c2d3f2cb88b None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:07.694951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca762ec-c5de-4dd5-9d3f-0c2d3f2cb88b None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:07.696482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca762ec-c5de-4dd5-9d3f-0c2d3f2cb88b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:07.698672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 196/388] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:07 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:07.704728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca762ec-c5de-4dd5-9d3f-0c2d3f2cb88b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:07.740699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc36dfd-11a8-4284-be18-70e574ee212e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:07.784492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10260340-cdea-4d5e-9e7f-c3a30f2a2dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:07.785585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10260340-cdea-4d5e-9e7f-c3a30f2a2dc9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:07.786029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10260340-cdea-4d5e-9e7f-c3a30f2a2dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:07.786471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10260340-cdea-4d5e-9e7f-c3a30f2a2dc9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:07.800634 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10260340-cdea-4d5e-9e7f-c3a30f2a2dc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-10260340-cdea-4d5e-9e7f-c3a30f2a2dc9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:07.812070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:07.813366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:07.814233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 193/389] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:41:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:07.815447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9ce53-60ce-4286-938b-a600159af4b5 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:07.815559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9ce53-60ce-4286-938b-a600159af4b5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:07.815767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9ce53-60ce-4286-938b-a600159af4b5 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:07.817404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 194/390] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:07.833121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a7d83-6ef4-4395-aa17-417c59a1fe24 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:07.833121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a7d83-6ef4-4395-aa17-417c59a1fe24 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:07.833121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a7d83-6ef4-4395-aa17-417c59a1fe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:07.848070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-117a7d83-6ef4-4395-aa17-417c59a1fe24 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:07.849957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-117a7d83-6ef4-4395-aa17-417c59a1fe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:07.870393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca762ec-c5de-4dd5-9d3f-0c2d3f2cb88b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:07.926751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40894945-d46f-4cab-9451-fc38f05847dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:07.928750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40894945-d46f-4cab-9451-fc38f05847dd None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:07.929508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40894945-d46f-4cab-9451-fc38f05847dd None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:07.930132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40894945-d46f-4cab-9451-fc38f05847dd None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd/groups/a7fc3dcf59e84e65b6807bebfd0f9987/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:07.933618 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40894945-d46f-4cab-9451-fc38f05847dd None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5716d18b14f8c83813becdf6695bd, group_id=a7fc3dcf59e84e65b6807bebfd0f9987, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:07.936579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40894945-d46f-4cab-9451-fc38f05847dd None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:07.962891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 197/391] 10.222.0.79 () {58 vars in 1442 bytes} [Fri May 13 13:41:07 2022] PUT /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd/groups/a7fc3dcf59e84e65b6807bebfd0f9987/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:41:07.964838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40894945-d46f-4cab-9451-fc38f05847dd None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:07.993585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 195/392] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:08.073227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a7d83-6ef4-4395-aa17-417c59a1fe24 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:08.160795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40894945-d46f-4cab-9451-fc38f05847dd None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:08.233683 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dff19-043f-4a41-bd54-c2828fb2979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:08.235511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dff19-043f-4a41-bd54-c2828fb2979a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:08.236322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dff19-043f-4a41-bd54-c2828fb2979a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:08.237077 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dff19-043f-4a41-bd54-c2828fb2979a None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:08.238762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dff19-043f-4a41-bd54-c2828fb2979a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:08.239702 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dff19-043f-4a41-bd54-c2828fb2979a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:08.244296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dff19-043f-4a41-bd54-c2828fb2979a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:08.264715 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 198/393] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:08.267602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6b5da-5edd-4cdc-be78-ef0c7daba0ad None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:08.268458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6b5da-5edd-4cdc-be78-ef0c7daba0ad None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:08.269179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6b5da-5edd-4cdc-be78-ef0c7daba0ad None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:08.272500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 199/394] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:08.277844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6b5da-5edd-4cdc-be78-ef0c7daba0ad None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:08.295348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a7d83-6ef4-4395-aa17-417c59a1fe24 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:08.337222 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6135f4-3b73-4101-b2fd-00051da2714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:08.338054 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6135f4-3b73-4101-b2fd-00051da2714a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:08.338335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6135f4-3b73-4101-b2fd-00051da2714a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:08.338609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6135f4-3b73-4101-b2fd-00051da2714a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:08.352296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6135f4-3b73-4101-b2fd-00051da2714a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-6b6135f4-3b73-4101-b2fd-00051da2714a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:08.361226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:08.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:08.363356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 196/395] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:41:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:08.364327 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb024b5-f6d6-49e3-ae8c-2ec32a1e3bae None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:08.364439 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb024b5-f6d6-49e3-ae8c-2ec32a1e3bae None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:08.370983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb024b5-f6d6-49e3-ae8c-2ec32a1e3bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:08.390538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-ddb024b5-f6d6-49e3-ae8c-2ec32a1e3bae None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:08.391762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ddb024b5-f6d6-49e3-ae8c-2ec32a1e3bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:08.426871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6b5da-5edd-4cdc-be78-ef0c7daba0ad None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:08.468635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 197/396] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:08.469812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb024b5-f6d6-49e3-ae8c-2ec32a1e3bae None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:08.482640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a33754-0a5a-452a-b378-d3b2736390a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:08.486820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33754-0a5a-452a-b378-d3b2736390a8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:08.486820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33754-0a5a-452a-b378-d3b2736390a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:08.486820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33754-0a5a-452a-b378-d3b2736390a8 None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:08.487121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a33754-0a5a-452a-b378-d3b2736390a8 None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:08.489747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a33754-0a5a-452a-b378-d3b2736390a8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:08.493588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 200/397] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:08 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:08.495053 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798909a-d766-48cf-8c89-3bda4c5ae716 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:08.495355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798909a-d766-48cf-8c89-3bda4c5ae716 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:08.495610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798909a-d766-48cf-8c89-3bda4c5ae716 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:08.497135 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 201/398] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:08.501792 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798909a-d766-48cf-8c89-3bda4c5ae716 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:08.607696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb024b5-f6d6-49e3-ae8c-2ec32a1e3bae None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:08.629550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798909a-d766-48cf-8c89-3bda4c5ae716 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:08.665853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee323bce-e4a4-4292-b82b-75cd67477018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:08.666972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee323bce-e4a4-4292-b82b-75cd67477018 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:08.667414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee323bce-e4a4-4292-b82b-75cd67477018 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:08.667824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee323bce-e4a4-4292-b82b-75cd67477018 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:08.669077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee323bce-e4a4-4292-b82b-75cd67477018 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:08.669575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee323bce-e4a4-4292-b82b-75cd67477018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:08.671802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee323bce-e4a4-4292-b82b-75cd67477018 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:08.690818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 198/399] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:08.690818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee323bce-e4a4-4292-b82b-75cd67477018 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:08.697358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d302fe3-ce5a-4eec-a168-68291f2ce74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:08.698232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d302fe3-ce5a-4eec-a168-68291f2ce74e None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:08.698513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d302fe3-ce5a-4eec-a168-68291f2ce74e None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:08.698898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d302fe3-ce5a-4eec-a168-68291f2ce74e None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:08.700198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d302fe3-ce5a-4eec-a168-68291f2ce74e None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:08.701638 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d302fe3-ce5a-4eec-a168-68291f2ce74e None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:08.703669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 202/400] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:08 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:08.704845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb18432-0b87-40ef-9d3b-13d8f6d03aad None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:08.705128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb18432-0b87-40ef-9d3b-13d8f6d03aad None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:08.705401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb18432-0b87-40ef-9d3b-13d8f6d03aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:08.726280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-7fb18432-0b87-40ef-9d3b-13d8f6d03aad None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:08.727388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7fb18432-0b87-40ef-9d3b-13d8f6d03aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:08.817773 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee323bce-e4a4-4292-b82b-75cd67477018 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:08.824488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 203/401] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:08.828856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb18432-0b87-40ef-9d3b-13d8f6d03aad None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:08.859409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8e5f8-557f-4d24-bc92-6dd4f57ef690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:08.860249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8e5f8-557f-4d24-bc92-6dd4f57ef690 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:08.860493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8e5f8-557f-4d24-bc92-6dd4f57ef690 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:08.860793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8e5f8-557f-4d24-bc92-6dd4f57ef690 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:08.876555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8e5f8-557f-4d24-bc92-6dd4f57ef690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-73d8e5f8-557f-4d24-bc92-6dd4f57ef690 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:08.892251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:08.893225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:08.894091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:08.894696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 199/402] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:08.895759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8e5f8-557f-4d24-bc92-6dd4f57ef690 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:08.951236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb18432-0b87-40ef-9d3b-13d8f6d03aad None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:09.000191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee89663-03b1-4fbe-b6dd-dbdb0c3c9a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:09.001498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee89663-03b1-4fbe-b6dd-dbdb0c3c9a6a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:09.001964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee89663-03b1-4fbe-b6dd-dbdb0c3c9a6a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:09.002432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee89663-03b1-4fbe-b6dd-dbdb0c3c9a6a None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:09.003801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee89663-03b1-4fbe-b6dd-dbdb0c3c9a6a None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:09.005247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee89663-03b1-4fbe-b6dd-dbdb0c3c9a6a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:09.006968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 204/403] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:08 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:09.007992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee89663-03b1-4fbe-b6dd-dbdb0c3c9a6a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:09.042006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8e5f8-557f-4d24-bc92-6dd4f57ef690 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:09.097915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb259946-31ed-4853-ae4a-88a1020bca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:09.099397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb259946-31ed-4853-ae4a-88a1020bca8d None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:09.099897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb259946-31ed-4853-ae4a-88a1020bca8d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:09.100406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb259946-31ed-4853-ae4a-88a1020bca8d None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de/groups/a7fc3dcf59e84e65b6807bebfd0f9987/roles/4501501962414ed0abfba693ba95918c` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:09.104323 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb259946-31ed-4853-ae4a-88a1020bca8d None admin] RBAC: Authorizing `identity:create_grant(project_id=e2682c48ad134e689170645b2c3b85de, group_id=a7fc3dcf59e84e65b6807bebfd0f9987, role_id=4501501962414ed0abfba693ba95918c)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:09.107417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb259946-31ed-4853-ae4a-88a1020bca8d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:09.123283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 200/404] 10.222.0.79 () {58 vars in 1442 bytes} [Fri May 13 13:41:08 2022] PUT /identity/v3/projects/e2682c48ad134e689170645b2c3b85de/groups/a7fc3dcf59e84e65b6807bebfd0f9987/roles/4501501962414ed0abfba693ba95918c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:41:09.124418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb259946-31ed-4853-ae4a-88a1020bca8d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:09.124822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee89663-03b1-4fbe-b6dd-dbdb0c3c9a6a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:09.166536 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c5401-c407-425b-ad0f-bc8efe118409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:09.169460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c5401-c407-425b-ad0f-bc8efe118409 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:09.170019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c5401-c407-425b-ad0f-bc8efe118409 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:09.170429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c5401-c407-425b-ad0f-bc8efe118409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:09.187063 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c5401-c407-425b-ad0f-bc8efe118409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-c85c5401-c407-425b-ad0f-bc8efe118409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:09.196115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:09.197158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:09.197942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:09.200651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 205/405] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:41:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:09.201842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c5401-c407-425b-ad0f-bc8efe118409 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:09.286911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb259946-31ed-4853-ae4a-88a1020bca8d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:09.333404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c5401-c407-425b-ad0f-bc8efe118409 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:09.339937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f4050-e12e-43bc-89c3-fe42afef7384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:09.340727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f4050-e12e-43bc-89c3-fe42afef7384 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:09.341075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f4050-e12e-43bc-89c3-fe42afef7384 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:09.341466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f4050-e12e-43bc-89c3-fe42afef7384 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:09.342293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f4050-e12e-43bc-89c3-fe42afef7384 None admin] RBAC: Adding query filter params (name=member) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:09.342668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f4050-e12e-43bc-89c3-fe42afef7384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:09.344891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f4050-e12e-43bc-89c3-fe42afef7384 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:09.356891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 201/406] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:41:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:09.358041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f4050-e12e-43bc-89c3-fe42afef7384 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:09.398324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a973dc-6c3e-4c15-866c-aea57f8b9405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:09.399337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a973dc-6c3e-4c15-866c-aea57f8b9405 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:09.401046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a973dc-6c3e-4c15-866c-aea57f8b9405 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:09.401570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a973dc-6c3e-4c15-866c-aea57f8b9405 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:09.404818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a973dc-6c3e-4c15-866c-aea57f8b9405 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:09.407551 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a973dc-6c3e-4c15-866c-aea57f8b9405 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:09.410617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 206/407] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:09 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:09.418782 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a973dc-6c3e-4c15-866c-aea57f8b9405 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:09.511310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f4050-e12e-43bc-89c3-fe42afef7384 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:09.549027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a973dc-6c3e-4c15-866c-aea57f8b9405 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:09.572095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af71e9-b149-4f7e-8bde-a9a1c51d24e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:09.573118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af71e9-b149-4f7e-8bde-a9a1c51d24e0 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:09.573626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af71e9-b149-4f7e-8bde-a9a1c51d24e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:09.574100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af71e9-b149-4f7e-8bde-a9a1c51d24e0 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:09.575137 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af71e9-b149-4f7e-8bde-a9a1c51d24e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:09.575763 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af71e9-b149-4f7e-8bde-a9a1c51d24e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:09.577710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af71e9-b149-4f7e-8bde-a9a1c51d24e0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:09.588585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 202/408] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:09.589807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af71e9-b149-4f7e-8bde-a9a1c51d24e0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:09.598740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c7b03-04ce-4e78-b955-095aa76a27a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:09.599722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c7b03-04ce-4e78-b955-095aa76a27a3 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:09.600121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c7b03-04ce-4e78-b955-095aa76a27a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:09.601387 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c7b03-04ce-4e78-b955-095aa76a27a3 None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:09.602264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c7b03-04ce-4e78-b955-095aa76a27a3 None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:09.603926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c7b03-04ce-4e78-b955-095aa76a27a3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:09.611380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 207/409] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:09 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:09.612357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c7b03-04ce-4e78-b955-095aa76a27a3 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:09.707407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af71e9-b149-4f7e-8bde-a9a1c51d24e0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:09.726772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c7b03-04ce-4e78-b955-095aa76a27a3 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:09.753453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea058e-c573-4b26-b359-0a83c700100d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:09.754352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea058e-c573-4b26-b359-0a83c700100d None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:09.754690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea058e-c573-4b26-b359-0a83c700100d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:09.754924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea058e-c573-4b26-b359-0a83c700100d None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd/groups/a7fc3dcf59e84e65b6807bebfd0f9987/roles/4501501962414ed0abfba693ba95918c` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:09.759926 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea058e-c573-4b26-b359-0a83c700100d None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5716d18b14f8c83813becdf6695bd, group_id=a7fc3dcf59e84e65b6807bebfd0f9987, role_id=4501501962414ed0abfba693ba95918c)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:09.761411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea058e-c573-4b26-b359-0a83c700100d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:09.781025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c51f76-3e0c-4ce5-bf46-928203e0ee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:09.782088 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c51f76-3e0c-4ce5-bf46-928203e0ee15 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:09.782370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c51f76-3e0c-4ce5-bf46-928203e0ee15 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:09.782644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c51f76-3e0c-4ce5-bf46-928203e0ee15 None admin] PATH_INFO: `/v3/groups/309505a759b04402a153d6d1086de51a` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:09.786065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 203/410] 10.222.0.79 () {58 vars in 1442 bytes} [Fri May 13 13:41:09 2022] PUT /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd/groups/a7fc3dcf59e84e65b6807bebfd0f9987/roles/4501501962414ed0abfba693ba95918c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:41:09.792678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea058e-c573-4b26-b359-0a83c700100d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:09.792842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c51f76-3e0c-4ce5-bf46-928203e0ee15 None admin] RBAC: Authorizing `identity:get_group(group_id=309505a759b04402a153d6d1086de51a)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:09.792842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c51f76-3e0c-4ce5-bf46-928203e0ee15 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:09.792842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 208/411] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:09 2022] GET /identity/v3/groups/309505a759b04402a153d6d1086de51a => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:09.802202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c51f76-3e0c-4ce5-bf46-928203e0ee15 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:09.973193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c51f76-3e0c-4ce5-bf46-928203e0ee15 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:09.982152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea058e-c573-4b26-b359-0a83c700100d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:10.013822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18fd2e-5989-4a1e-a89f-b12ba8333782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:10.014746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18fd2e-5989-4a1e-a89f-b12ba8333782 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:10.015080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18fd2e-5989-4a1e-a89f-b12ba8333782 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:10.015392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18fd2e-5989-4a1e-a89f-b12ba8333782 None admin] PATH_INFO: `/v3/projects/584c523555e7486e977f27ddcc51047b` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:10.019721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18fd2e-5989-4a1e-a89f-b12ba8333782 None admin] RBAC: Authorizing `identity:get_project(project_id=584c523555e7486e977f27ddcc51047b)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:10.021231 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18fd2e-5989-4a1e-a89f-b12ba8333782 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:10.022125 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd9416-6d13-4fd1-af5e-9768003ddb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:10.023358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd9416-6d13-4fd1-af5e-9768003ddb75 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:10.023595 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 209/412] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:09 2022] GET /identity/v3/projects/584c523555e7486e977f27ddcc51047b => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:10.024065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd9416-6d13-4fd1-af5e-9768003ddb75 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:10.024542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd9416-6d13-4fd1-af5e-9768003ddb75 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:10.031184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd9416-6d13-4fd1-af5e-9768003ddb75 None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:10.031375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18fd2e-5989-4a1e-a89f-b12ba8333782 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:10.033494 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd9416-6d13-4fd1-af5e-9768003ddb75 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:10.036583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 204/413] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:09 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:10.043499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd9416-6d13-4fd1-af5e-9768003ddb75 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:10.192249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18fd2e-5989-4a1e-a89f-b12ba8333782 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:10.210486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd9416-6d13-4fd1-af5e-9768003ddb75 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:10.232782 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb03e1-3d23-44b0-94ad-2025ebd4e2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:10.233780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb03e1-3d23-44b0-94ad-2025ebd4e2c9 None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:10.234159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb03e1-3d23-44b0-94ad-2025ebd4e2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:10.234520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb03e1-3d23-44b0-94ad-2025ebd4e2c9 None admin] PATH_INFO: `/v3/projects/584c523555e7486e977f27ddcc51047b/groups/309505a759b04402a153d6d1086de51a/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:10.237028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb03e1-3d23-44b0-94ad-2025ebd4e2c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=584c523555e7486e977f27ddcc51047b, group_id=309505a759b04402a153d6d1086de51a, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:10.238580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb03e1-3d23-44b0-94ad-2025ebd4e2c9 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:10.259510 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 210/414] 10.222.0.79 () {58 vars in 1442 bytes} [Fri May 13 13:41:10 2022] PUT /identity/v3/projects/584c523555e7486e977f27ddcc51047b/groups/309505a759b04402a153d6d1086de51a/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:41:10.274114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322b383-cc5c-4692-9f7c-40dc997b3924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:10.275220 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322b383-cc5c-4692-9f7c-40dc997b3924 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:10.275489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322b383-cc5c-4692-9f7c-40dc997b3924 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:10.275815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322b383-cc5c-4692-9f7c-40dc997b3924 None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de/groups/a7fc3dcf59e84e65b6807bebfd0f9987/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:10.278199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322b383-cc5c-4692-9f7c-40dc997b3924 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2682c48ad134e689170645b2c3b85de, group_id=a7fc3dcf59e84e65b6807bebfd0f9987, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:10.280121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322b383-cc5c-4692-9f7c-40dc997b3924 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:10.298990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 205/415] 10.222.0.79 () {58 vars in 1442 bytes} [Fri May 13 13:41:10 2022] PUT /identity/v3/projects/e2682c48ad134e689170645b2c3b85de/groups/a7fc3dcf59e84e65b6807bebfd0f9987/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:41:11.152672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe14f5-c365-4506-a865-6ab45f49be8c None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:11.153449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe14f5-c365-4506-a865-6ab45f49be8c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:11.153592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe14f5-c365-4506-a865-6ab45f49be8c None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:11.155479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 211/416] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:11.165156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba3b28-4c74-4243-986b-17c90bcb89ed None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:11.166434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba3b28-4c74-4243-986b-17c90bcb89ed None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:11.167150 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba3b28-4c74-4243-986b-17c90bcb89ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:11.193484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-6eba3b28-4c74-4243-986b-17c90bcb89ed None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:11.195633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6eba3b28-4c74-4243-986b-17c90bcb89ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:11.290580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 206/417] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:11.340962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe14f5-c365-4506-a865-6ab45f49be8c None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:11.536600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe14f5-c365-4506-a865-6ab45f49be8c None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:11.580079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc97e1-afce-428a-8217-b670d3f0cab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:11.581159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc97e1-afce-428a-8217-b670d3f0cab1 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:11.581540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc97e1-afce-428a-8217-b670d3f0cab1 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:11.581917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc97e1-afce-428a-8217-b670d3f0cab1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:11.603889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc97e1-afce-428a-8217-b670d3f0cab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-0fcc97e1-afce-428a-8217-b670d3f0cab1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:11.611861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:11.612783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:11.613563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:11.618525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 212/418] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:11.620144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba3b28-4c74-4243-986b-17c90bcb89ed None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:11.900804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba3b28-4c74-4243-986b-17c90bcb89ed None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:11.945357 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426f401-df9e-4baf-adac-28359821e715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:11.946109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426f401-df9e-4baf-adac-28359821e715 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:11.946462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426f401-df9e-4baf-adac-28359821e715 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:11.946758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426f401-df9e-4baf-adac-28359821e715 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:11.947517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426f401-df9e-4baf-adac-28359821e715 None admin] RBAC: Adding query filter params (name=member) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:11.947825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426f401-df9e-4baf-adac-28359821e715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:11.949247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426f401-df9e-4baf-adac-28359821e715 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:11.964497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 207/419] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:41:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:11.971889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc97e1-afce-428a-8217-b670d3f0cab1 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:12.185764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc97e1-afce-428a-8217-b670d3f0cab1 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:12.225884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b07dd-82a3-4b3a-a07c-278eba798290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:12.226974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b07dd-82a3-4b3a-a07c-278eba798290 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:12.227453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b07dd-82a3-4b3a-a07c-278eba798290 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:12.227922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b07dd-82a3-4b3a-a07c-278eba798290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:12.253008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43174ed4-8fc1-430d-b6b4-2b16c058c87a None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:12.254052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43174ed4-8fc1-430d-b6b4-2b16c058c87a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:12.254574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43174ed4-8fc1-430d-b6b4-2b16c058c87a None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:12.257100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 208/420] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:12.268280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d241a9-0f44-4366-9a76-92b91a918866 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:12.268932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d241a9-0f44-4366-9a76-92b91a918866 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:12.269423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d241a9-0f44-4366-9a76-92b91a918866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:12.303402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-68d241a9-0f44-4366-9a76-92b91a918866 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:12.305124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-68d241a9-0f44-4366-9a76-92b91a918866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:12.308072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b07dd-82a3-4b3a-a07c-278eba798290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:12.309569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b07dd-82a3-4b3a-a07c-278eba798290 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:12.424145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 213/421] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:12.436008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b07dd-82a3-4b3a-a07c-278eba798290 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:12.438545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 209/422] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:12.519232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d241a9-0f44-4366-9a76-92b91a918866 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:12.546377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b07dd-82a3-4b3a-a07c-278eba798290 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:12.589917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8527a1b-e7c4-40ac-ac90-5eeb48f5a607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:12.592071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8527a1b-e7c4-40ac-ac90-5eeb48f5a607 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:12.592715 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8527a1b-e7c4-40ac-ac90-5eeb48f5a607 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:12.594559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8527a1b-e7c4-40ac-ac90-5eeb48f5a607 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:12.596909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8527a1b-e7c4-40ac-ac90-5eeb48f5a607 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:12.599999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8527a1b-e7c4-40ac-ac90-5eeb48f5a607 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:12.604157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 214/423] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:12 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:12.613093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8527a1b-e7c4-40ac-ac90-5eeb48f5a607 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:12.764500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8527a1b-e7c4-40ac-ac90-5eeb48f5a607 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:12.798920 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d241a9-0f44-4366-9a76-92b91a918866 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:12.815558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1c0a6-ffc9-4ac1-868c-2e0b2f247b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:12.816444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1c0a6-ffc9-4ac1-868c-2e0b2f247b75 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:12.816722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1c0a6-ffc9-4ac1-868c-2e0b2f247b75 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:12.816993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1c0a6-ffc9-4ac1-868c-2e0b2f247b75 None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:12.820062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1c0a6-ffc9-4ac1-868c-2e0b2f247b75 None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:12.821583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1c0a6-ffc9-4ac1-868c-2e0b2f247b75 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:12.823711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 215/424] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:12 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:12.830171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1c0a6-ffc9-4ac1-868c-2e0b2f247b75 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:12.864572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725c19c-ecc6-4edd-af9f-0829598d5457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:12.865410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725c19c-ecc6-4edd-af9f-0829598d5457 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:12.866375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725c19c-ecc6-4edd-af9f-0829598d5457 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:12.866651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725c19c-ecc6-4edd-af9f-0829598d5457 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:12.893172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725c19c-ecc6-4edd-af9f-0829598d5457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-6725c19c-ecc6-4edd-af9f-0829598d5457 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:12.901599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:12.902918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:12.903855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 210/425] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:41:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:12.907695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7018af6-808d-4e48-9cb4-c6cf22e420b8 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:12.907695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7018af6-808d-4e48-9cb4-c6cf22e420b8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:12.907695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7018af6-808d-4e48-9cb4-c6cf22e420b8 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:12.907695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 211/426] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:12.909215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7018af6-808d-4e48-9cb4-c6cf22e420b8 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:13.009579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1c0a6-ffc9-4ac1-868c-2e0b2f247b75 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:13.062345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f077481-a72d-4b66-8bd9-86f9bae6a8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:13.062953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f077481-a72d-4b66-8bd9-86f9bae6a8da None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.063232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f077481-a72d-4b66-8bd9-86f9bae6a8da None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.063500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f077481-a72d-4b66-8bd9-86f9bae6a8da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.064342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f077481-a72d-4b66-8bd9-86f9bae6a8da None admin] RBAC: Adding query filter params (group.id=a7fc3dcf59e84e65b6807bebfd0f9987, role.id=90868071f00340ae853f1188777a2244, scope.project.id=8ff5716d18b14f8c83813becdf6695bd) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:13.064647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f077481-a72d-4b66-8bd9-86f9bae6a8da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:13.066112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f077481-a72d-4b66-8bd9-86f9bae6a8da None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:13.068572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7018af6-808d-4e48-9cb4-c6cf22e420b8 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:13.073447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 216/427] 10.222.0.79 () {56 vars in 1376 bytes} [Fri May 13 13:41:12 2022] GET /identity/v3/role_assignments?group.id=a7fc3dcf59e84e65b6807bebfd0f9987&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=90868071f00340ae853f1188777a2244 => generated 611 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:13.075020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1937352-7d2a-4e52-825c-36b9088d4688 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.075300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1937352-7d2a-4e52-825c-36b9088d4688 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.075572 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1937352-7d2a-4e52-825c-36b9088d4688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.097078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e1937352-7d2a-4e52-825c-36b9088d4688 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:13.098630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e1937352-7d2a-4e52-825c-36b9088d4688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:13.119625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91929bcb-407a-480d-add1-7364e704fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:13.120343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929bcb-407a-480d-add1-7364e704fb7f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.120740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929bcb-407a-480d-add1-7364e704fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.121093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91929bcb-407a-480d-add1-7364e704fb7f None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.121943 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929bcb-407a-480d-add1-7364e704fb7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:13.122254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929bcb-407a-480d-add1-7364e704fb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:13.123739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91929bcb-407a-480d-add1-7364e704fb7f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:13.136825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 212/428] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:13.147384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91929bcb-407a-480d-add1-7364e704fb7f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:13.240675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 217/429] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:13.275863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1937352-7d2a-4e52-825c-36b9088d4688 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:13.376273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91929bcb-407a-480d-add1-7364e704fb7f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:13.421904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4a86c-ead7-4549-bdeb-b3911fc2ea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:13.422864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4a86c-ead7-4549-bdeb-b3911fc2ea2e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.423156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4a86c-ead7-4549-bdeb-b3911fc2ea2e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.423430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4a86c-ead7-4549-bdeb-b3911fc2ea2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.435401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1937352-7d2a-4e52-825c-36b9088d4688 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:13.481780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6733e-21fd-49a2-be0c-f8a8337a4be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:13.483826 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6733e-21fd-49a2-be0c-f8a8337a4be8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.484295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6733e-21fd-49a2-be0c-f8a8337a4be8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.484643 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6733e-21fd-49a2-be0c-f8a8337a4be8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.497090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6733e-21fd-49a2-be0c-f8a8337a4be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-20b6733e-21fd-49a2-be0c-f8a8337a4be8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:13.505897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:13.508422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:13.510501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 218/430] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:41:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:13.514772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4a86c-ead7-4549-bdeb-b3911fc2ea2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:13.514772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4a86c-ead7-4549-bdeb-b3911fc2ea2e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:13.516085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6733e-21fd-49a2-be0c-f8a8337a4be8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:13.642278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 213/431] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:41:13.646802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae5ac-f4e3-425b-9c01-83eaa77b2700 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.646802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae5ac-f4e3-425b-9c01-83eaa77b2700 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.646802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae5ac-f4e3-425b-9c01-83eaa77b2700 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.646802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 214/432] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:13.649804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae5ac-f4e3-425b-9c01-83eaa77b2700 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:13.662885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6733e-21fd-49a2-be0c-f8a8337a4be8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:13.711072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095bfa0-018b-446d-b1a6-d48a10ee0069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:13.712222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095bfa0-018b-446d-b1a6-d48a10ee0069 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.712498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095bfa0-018b-446d-b1a6-d48a10ee0069 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.712778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095bfa0-018b-446d-b1a6-d48a10ee0069 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.713538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095bfa0-018b-446d-b1a6-d48a10ee0069 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:13.713844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095bfa0-018b-446d-b1a6-d48a10ee0069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:13.716027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095bfa0-018b-446d-b1a6-d48a10ee0069 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:13.724954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 219/433] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:13.726898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb599010-da32-45a6-9768-6c1fb6a967e0 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.726898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb599010-da32-45a6-9768-6c1fb6a967e0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.730914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb599010-da32-45a6-9768-6c1fb6a967e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.759399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-fb599010-da32-45a6-9768-6c1fb6a967e0 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:13.761297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-fb599010-da32-45a6-9768-6c1fb6a967e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:13.810122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae5ac-f4e3-425b-9c01-83eaa77b2700 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:13.863924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0547e-e0de-4a7b-a502-c2b2315045ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:13.865316 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0547e-e0de-4a7b-a502-c2b2315045ad None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.865805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0547e-e0de-4a7b-a502-c2b2315045ad None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.866252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0547e-e0de-4a7b-a502-c2b2315045ad None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.878986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 220/434] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:13.879177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0547e-e0de-4a7b-a502-c2b2315045ad None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:13.879261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b3299-6ce4-42e4-8420-532c4c212b03 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.879261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b3299-6ce4-42e4-8420-532c4c212b03 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.879261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b3299-6ce4-42e4-8420-532c4c212b03 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.879470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0547e-e0de-4a7b-a502-c2b2315045ad None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:13.879547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 221/435] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:13.879547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b3299-6ce4-42e4-8420-532c4c212b03 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:13.879682 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 215/436] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:13 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:13.879682 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31290c-1bb9-467e-89c4-e01f4863fd46 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:13.879682 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31290c-1bb9-467e-89c4-e01f4863fd46 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:13.879857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31290c-1bb9-467e-89c4-e01f4863fd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:13.897741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-9c31290c-1bb9-467e-89c4-e01f4863fd46 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:13.898919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9c31290c-1bb9-467e-89c4-e01f4863fd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:13.998870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 216/437] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:13.998870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31290c-1bb9-467e-89c4-e01f4863fd46 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:14.027518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b3299-6ce4-42e4-8420-532c4c212b03 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:14.064244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e0cc5-3a58-443e-9f56-363e14168356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:14.064911 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0cc5-3a58-443e-9f56-363e14168356 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:14.065188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0cc5-3a58-443e-9f56-363e14168356 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:14.065460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0cc5-3a58-443e-9f56-363e14168356 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:14.108645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31290c-1bb9-467e-89c4-e01f4863fd46 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:14.141394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e0cc5-3a58-443e-9f56-363e14168356 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:14.142957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e0cc5-3a58-443e-9f56-363e14168356 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:14.151623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fe87c-a3d1-42f3-8dde-1a4fea6a678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:14.152430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fe87c-a3d1-42f3-8dde-1a4fea6a678e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:14.152706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fe87c-a3d1-42f3-8dde-1a4fea6a678e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:14.152978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fe87c-a3d1-42f3-8dde-1a4fea6a678e None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:14.154236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fe87c-a3d1-42f3-8dde-1a4fea6a678e None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:14.155758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fe87c-a3d1-42f3-8dde-1a4fea6a678e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:14.157441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 217/438] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:13 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:14.157441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fe87c-a3d1-42f3-8dde-1a4fea6a678e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:14.258941 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 222/439] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:14.259766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e0cc5-3a58-443e-9f56-363e14168356 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:14.273114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fe87c-a3d1-42f3-8dde-1a4fea6a678e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:14.309434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8ce53-8b87-4e49-afb1-6e5b25dffc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:14.310397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8ce53-8b87-4e49-afb1-6e5b25dffc8c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:14.310694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8ce53-8b87-4e49-afb1-6e5b25dffc8c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:14.311020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8ce53-8b87-4e49-afb1-6e5b25dffc8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:14.322723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8ce53-8b87-4e49-afb1-6e5b25dffc8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-65d8ce53-8b87-4e49-afb1-6e5b25dffc8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:14.330834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:14.331968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 218/440] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:41:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:14.332885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8ce53-8b87-4e49-afb1-6e5b25dffc8c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:14.367620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e0cc5-3a58-443e-9f56-363e14168356 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:14.406483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d9442-5de4-49ed-80de-d77d3f28c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:14.407268 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d9442-5de4-49ed-80de-d77d3f28c500 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:14.407531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d9442-5de4-49ed-80de-d77d3f28c500 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:14.407800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d9442-5de4-49ed-80de-d77d3f28c500 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:14.418434 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d9442-5de4-49ed-80de-d77d3f28c500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-e89d9442-5de4-49ed-80de-d77d3f28c500 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:14.426189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:14.427465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 223/441] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:14.428269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d9442-5de4-49ed-80de-d77d3f28c500 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:14.447880 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8ce53-8b87-4e49-afb1-6e5b25dffc8c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:14.485050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6219d-55d9-469e-97b7-de44740459e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:14.485738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6219d-55d9-469e-97b7-de44740459e4 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:14.486025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6219d-55d9-469e-97b7-de44740459e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:14.486314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6219d-55d9-469e-97b7-de44740459e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:14.487238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6219d-55d9-469e-97b7-de44740459e4 None admin] RBAC: Adding query filter params (group.id=a7fc3dcf59e84e65b6807bebfd0f9987, role.id=4501501962414ed0abfba693ba95918c, scope.project.id=e2682c48ad134e689170645b2c3b85de) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:14.487553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6219d-55d9-469e-97b7-de44740459e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:14.489059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6219d-55d9-469e-97b7-de44740459e4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:14.497262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 219/442] 10.222.0.79 () {56 vars in 1376 bytes} [Fri May 13 13:41:14 2022] GET /identity/v3/role_assignments?group.id=a7fc3dcf59e84e65b6807bebfd0f9987&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=4501501962414ed0abfba693ba95918c => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:14.498257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6219d-55d9-469e-97b7-de44740459e4 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:14.560170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d9442-5de4-49ed-80de-d77d3f28c500 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:14.612277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582653e8-4aeb-402f-8ee4-71719017aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:14.613372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582653e8-4aeb-402f-8ee4-71719017aed0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:14.613707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582653e8-4aeb-402f-8ee4-71719017aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:14.613953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582653e8-4aeb-402f-8ee4-71719017aed0 None admin] PATH_INFO: `/v3/projects/8ff5716d18b14f8c83813becdf6695bd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:14.615521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582653e8-4aeb-402f-8ee4-71719017aed0 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5716d18b14f8c83813becdf6695bd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:14.617135 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582653e8-4aeb-402f-8ee4-71719017aed0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:14.624307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 224/443] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:14 2022] GET /identity/v3/projects/8ff5716d18b14f8c83813becdf6695bd => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:14.624307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582653e8-4aeb-402f-8ee4-71719017aed0 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:14.662345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6219d-55d9-469e-97b7-de44740459e4 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:14.707976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370a6b8-5916-4c71-99d5-5ef365e8faea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:14.709246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370a6b8-5916-4c71-99d5-5ef365e8faea None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:14.709746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370a6b8-5916-4c71-99d5-5ef365e8faea None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:14.710186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370a6b8-5916-4c71-99d5-5ef365e8faea None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:14.711033 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370a6b8-5916-4c71-99d5-5ef365e8faea None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:14.711349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370a6b8-5916-4c71-99d5-5ef365e8faea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:14.713399 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370a6b8-5916-4c71-99d5-5ef365e8faea None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:14.722261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 220/444] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:14.723390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370a6b8-5916-4c71-99d5-5ef365e8faea None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:14.757920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582653e8-4aeb-402f-8ee4-71719017aed0 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:14.814831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b92d97-431b-4a48-936d-6d2a87ffd175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:14.815515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b92d97-431b-4a48-936d-6d2a87ffd175 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:14.815787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b92d97-431b-4a48-936d-6d2a87ffd175 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:14.816057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b92d97-431b-4a48-936d-6d2a87ffd175 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:14.816809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b92d97-431b-4a48-936d-6d2a87ffd175 None admin] RBAC: Adding query filter params (name=member) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:14.817111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b92d97-431b-4a48-936d-6d2a87ffd175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:14.819291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b92d97-431b-4a48-936d-6d2a87ffd175 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:14.828720 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 225/445] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:41:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:14.829754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b92d97-431b-4a48-936d-6d2a87ffd175 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:14.858473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370a6b8-5916-4c71-99d5-5ef365e8faea None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:14.895615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36995c83-615c-492a-a566-fcdf64552626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:14.896508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36995c83-615c-492a-a566-fcdf64552626 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:14.896826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36995c83-615c-492a-a566-fcdf64552626 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:14.897101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36995c83-615c-492a-a566-fcdf64552626 None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:14.898569 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36995c83-615c-492a-a566-fcdf64552626 None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:14.900107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36995c83-615c-492a-a566-fcdf64552626 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:14.901809 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 221/446] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:14 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:14.902640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36995c83-615c-492a-a566-fcdf64552626 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:14.951252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b92d97-431b-4a48-936d-6d2a87ffd175 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:14.987330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd8b38-f3ff-446d-bf8c-8e796489ee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:14.987993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd8b38-f3ff-446d-bf8c-8e796489ee48 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:14.988295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd8b38-f3ff-446d-bf8c-8e796489ee48 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:14.988528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd8b38-f3ff-446d-bf8c-8e796489ee48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:15.018452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36995c83-615c-492a-a566-fcdf64552626 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:15.055016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601194f-ee64-48a0-9abc-4a8ca8bbaa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:15.055685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601194f-ee64-48a0-9abc-4a8ca8bbaa0f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:15.055992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601194f-ee64-48a0-9abc-4a8ca8bbaa0f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:15.056273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601194f-ee64-48a0-9abc-4a8ca8bbaa0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:15.062267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd8b38-f3ff-446d-bf8c-8e796489ee48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:15.063855 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd8b38-f3ff-446d-bf8c-8e796489ee48 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:15.130768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601194f-ee64-48a0-9abc-4a8ca8bbaa0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:15.132394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601194f-ee64-48a0-9abc-4a8ca8bbaa0f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:15.179647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 226/447] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:15.180527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd8b38-f3ff-446d-bf8c-8e796489ee48 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:15.248182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 222/448] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:15.249058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601194f-ee64-48a0-9abc-4a8ca8bbaa0f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:15.298576 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd8b38-f3ff-446d-bf8c-8e796489ee48 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:15.335996 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0b36c-569a-41dc-8c68-0d807b67a2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:15.336652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0b36c-569a-41dc-8c68-0d807b67a2c2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:15.336929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0b36c-569a-41dc-8c68-0d807b67a2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:15.337201 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0b36c-569a-41dc-8c68-0d807b67a2c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:15.338030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0b36c-569a-41dc-8c68-0d807b67a2c2 None admin] RBAC: Adding query filter params (group.id=a7fc3dcf59e84e65b6807bebfd0f9987, role.id=4501501962414ed0abfba693ba95918c, scope.project.id=8ff5716d18b14f8c83813becdf6695bd) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:15.338337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0b36c-569a-41dc-8c68-0d807b67a2c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:15.339820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0b36c-569a-41dc-8c68-0d807b67a2c2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:15.348198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 227/449] 10.222.0.79 () {56 vars in 1376 bytes} [Fri May 13 13:41:15 2022] GET /identity/v3/role_assignments?group.id=a7fc3dcf59e84e65b6807bebfd0f9987&scope.project.id=8ff5716d18b14f8c83813becdf6695bd&role.id=4501501962414ed0abfba693ba95918c => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:15.349517 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0b36c-569a-41dc-8c68-0d807b67a2c2 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:15.366146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601194f-ee64-48a0-9abc-4a8ca8bbaa0f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:15.409637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225511-e957-4bb8-a4db-93f60e9411e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:15.410549 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225511-e957-4bb8-a4db-93f60e9411e6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:15.410865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225511-e957-4bb8-a4db-93f60e9411e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:15.411142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2225511-e957-4bb8-a4db-93f60e9411e6 None admin] PATH_INFO: `/v3/projects/584c523555e7486e977f27ddcc51047b` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:15.413312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2225511-e957-4bb8-a4db-93f60e9411e6 None admin] RBAC: Authorizing `identity:get_project(project_id=584c523555e7486e977f27ddcc51047b)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:15.414975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2225511-e957-4bb8-a4db-93f60e9411e6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:15.417005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 223/450] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:15 2022] GET /identity/v3/projects/584c523555e7486e977f27ddcc51047b => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:15.423511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225511-e957-4bb8-a4db-93f60e9411e6 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:15.499267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0b36c-569a-41dc-8c68-0d807b67a2c2 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:15.553809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45563872-b6e1-46b6-86d4-2a734c45279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:15.555149 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563872-b6e1-46b6-86d4-2a734c45279d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:15.555660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563872-b6e1-46b6-86d4-2a734c45279d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:15.556136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45563872-b6e1-46b6-86d4-2a734c45279d None admin] PATH_INFO: `/v3/projects/e2682c48ad134e689170645b2c3b85de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:15.557885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45563872-b6e1-46b6-86d4-2a734c45279d None admin] RBAC: Authorizing `identity:get_project(project_id=e2682c48ad134e689170645b2c3b85de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:15.559656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45563872-b6e1-46b6-86d4-2a734c45279d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:15.561681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 228/451] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:41:15 2022] GET /identity/v3/projects/e2682c48ad134e689170645b2c3b85de => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:15.569043 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45563872-b6e1-46b6-86d4-2a734c45279d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:15.577178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2225511-e957-4bb8-a4db-93f60e9411e6 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:15.627869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced891a5-b1e8-4206-9840-08ca1e70cfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:15.628978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced891a5-b1e8-4206-9840-08ca1e70cfe6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:15.629121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced891a5-b1e8-4206-9840-08ca1e70cfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:15.629351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced891a5-b1e8-4206-9840-08ca1e70cfe6 None admin] PATH_INFO: `/v3/groups/309505a759b04402a153d6d1086de51a` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:15.630778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced891a5-b1e8-4206-9840-08ca1e70cfe6 None admin] RBAC: Authorizing `identity:get_group(group_id=309505a759b04402a153d6d1086de51a)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:15.632235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced891a5-b1e8-4206-9840-08ca1e70cfe6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:15.633960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 224/452] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:15 2022] GET /identity/v3/groups/309505a759b04402a153d6d1086de51a => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:15.640223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced891a5-b1e8-4206-9840-08ca1e70cfe6 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:15.715990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45563872-b6e1-46b6-86d4-2a734c45279d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:15.772827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51090c-7907-4b62-b608-78c386b2300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:15.773626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51090c-7907-4b62-b608-78c386b2300d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:15.774643 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51090c-7907-4b62-b608-78c386b2300d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:15.774643 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51090c-7907-4b62-b608-78c386b2300d None admin] PATH_INFO: `/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:15.775662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51090c-7907-4b62-b608-78c386b2300d None admin] RBAC: Authorizing `identity:get_group(group_id=a7fc3dcf59e84e65b6807bebfd0f9987)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:15.777158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51090c-7907-4b62-b608-78c386b2300d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:15.780271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 229/453] 10.222.0.79 () {56 vars in 1180 bytes} [Fri May 13 13:41:15 2022] GET /identity/v3/groups/a7fc3dcf59e84e65b6807bebfd0f9987 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:15.782037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced891a5-b1e8-4206-9840-08ca1e70cfe6 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:15.788941 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51090c-7907-4b62-b608-78c386b2300d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:15.828160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498ae14-180d-48f3-8ac3-93b447f50218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:15.828837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498ae14-180d-48f3-8ac3-93b447f50218 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:15.829125 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498ae14-180d-48f3-8ac3-93b447f50218 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:15.829402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498ae14-180d-48f3-8ac3-93b447f50218 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:15.830242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498ae14-180d-48f3-8ac3-93b447f50218 None admin] RBAC: Adding query filter params (group.id=309505a759b04402a153d6d1086de51a, role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=584c523555e7486e977f27ddcc51047b) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:15.830552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498ae14-180d-48f3-8ac3-93b447f50218 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:15.832074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498ae14-180d-48f3-8ac3-93b447f50218 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:15.838994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 225/454] 10.222.0.79 () {56 vars in 1376 bytes} [Fri May 13 13:41:15 2022] GET /identity/v3/role_assignments?group.id=309505a759b04402a153d6d1086de51a&scope.project.id=584c523555e7486e977f27ddcc51047b&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:15.927160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51090c-7907-4b62-b608-78c386b2300d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:15.965570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589cd70-970f-4bd7-9685-8926365b6539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:15.966239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589cd70-970f-4bd7-9685-8926365b6539 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:15.966519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589cd70-970f-4bd7-9685-8926365b6539 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:15.966833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589cd70-970f-4bd7-9685-8926365b6539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:15.967672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589cd70-970f-4bd7-9685-8926365b6539 None admin] RBAC: Adding query filter params (group.id=a7fc3dcf59e84e65b6807bebfd0f9987, role.id=90868071f00340ae853f1188777a2244, scope.project.id=e2682c48ad134e689170645b2c3b85de) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:15.967976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589cd70-970f-4bd7-9685-8926365b6539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:15.969444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589cd70-970f-4bd7-9685-8926365b6539 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:15.976937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 230/455] 10.222.0.79 () {56 vars in 1376 bytes} [Fri May 13 13:41:15 2022] GET /identity/v3/role_assignments?group.id=a7fc3dcf59e84e65b6807bebfd0f9987&scope.project.id=e2682c48ad134e689170645b2c3b85de&role.id=90868071f00340ae853f1188777a2244 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:19.507746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188ee49-d39e-4eb2-912e-b57bd1a43016 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:19.508315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188ee49-d39e-4eb2-912e-b57bd1a43016 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:19.508633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188ee49-d39e-4eb2-912e-b57bd1a43016 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:19.511611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 226/456] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:19.517222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f48daa-0a35-428c-a50f-701027b93100 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:19.518233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f48daa-0a35-428c-a50f-701027b93100 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:19.518781 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f48daa-0a35-428c-a50f-701027b93100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:19.547957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-52f48daa-0a35-428c-a50f-701027b93100 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:19.549539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-52f48daa-0a35-428c-a50f-701027b93100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:19.657495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 231/457] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:19.694672 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188ee49-d39e-4eb2-912e-b57bd1a43016 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:19.765301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00082c9c-a076-4568-8438-d651d91ea7f5 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:19.765957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00082c9c-a076-4568-8438-d651d91ea7f5 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:19.766452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00082c9c-a076-4568-8438-d651d91ea7f5 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:19.768406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 232/458] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:19.774115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4211c6a-2c58-4511-891d-95fda6acca67 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:19.774761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4211c6a-2c58-4511-891d-95fda6acca67 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:19.775390 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4211c6a-2c58-4511-891d-95fda6acca67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:19.811372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a4211c6a-2c58-4511-891d-95fda6acca67 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:19.813866 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a4211c6a-2c58-4511-891d-95fda6acca67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:19.907753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188ee49-d39e-4eb2-912e-b57bd1a43016 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:19.943650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 233/459] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:19.946427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d7175-18d4-42e7-a50c-09ac30d54048 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:19.947228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d7175-18d4-42e7-a50c-09ac30d54048 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:19.947970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d7175-18d4-42e7-a50c-09ac30d54048 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:19.951056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 234/460] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:19.957915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5af4f1-055f-43b8-90c1-b8e16e585560 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:19.959812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5af4f1-055f-43b8-90c1-b8e16e585560 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:19.960952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5af4f1-055f-43b8-90c1-b8e16e585560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:19.975611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29482e1-8a0d-4f43-bc47-1159e6695534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:19.975611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29482e1-8a0d-4f43-bc47-1159e6695534 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:19.975611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29482e1-8a0d-4f43-bc47-1159e6695534 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:19.975611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29482e1-8a0d-4f43-bc47-1159e6695534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:19.995638 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-be5af4f1-055f-43b8-90c1-b8e16e585560 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:19.995820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29482e1-8a0d-4f43-bc47-1159e6695534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:19.997427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-be5af4f1-055f-43b8-90c1-b8e16e585560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:19.998313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29482e1-8a0d-4f43-bc47-1159e6695534 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-c29482e1-8a0d-4f43-bc47-1159e6695534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:20.038593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:20.040112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:20.040960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:20.040960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:20.040960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:20.040960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:20.041179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 227/461] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:20.042457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29482e1-8a0d-4f43-bc47-1159e6695534 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:20.111929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 235/462] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:20.113442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5af4f1-055f-43b8-90c1-b8e16e585560 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:20.177577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29482e1-8a0d-4f43-bc47-1159e6695534 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:20.215359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87ae36-d564-4399-8c87-5327177cf3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:20.216560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87ae36-d564-4399-8c87-5327177cf3e0 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:20.217043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87ae36-d564-4399-8c87-5327177cf3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:20.217516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87ae36-d564-4399-8c87-5327177cf3e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:20.229940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87ae36-d564-4399-8c87-5327177cf3e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:20.231811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87ae36-d564-4399-8c87-5327177cf3e0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-6e87ae36-d564-4399-8c87-5327177cf3e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:20.252057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:20.252954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:20.253881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:20.253881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:20.253881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:20.253881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:20.257189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 228/463] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:20.258407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87ae36-d564-4399-8c87-5327177cf3e0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:20.272409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5af4f1-055f-43b8-90c1-b8e16e585560 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:20.318141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86eab0-a3fd-40aa-87af-eabbe46441ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:20.318882 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86eab0-a3fd-40aa-87af-eabbe46441ab None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:20.319129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86eab0-a3fd-40aa-87af-eabbe46441ab None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:20.319415 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86eab0-a3fd-40aa-87af-eabbe46441ab None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:20.320138 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86eab0-a3fd-40aa-87af-eabbe46441ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:20.320444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86eab0-a3fd-40aa-87af-eabbe46441ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:20.321846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86eab0-a3fd-40aa-87af-eabbe46441ab None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:20.341147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 236/464] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:20.342090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86eab0-a3fd-40aa-87af-eabbe46441ab None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:20.375983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87ae36-d564-4399-8c87-5327177cf3e0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:20.417019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98383ad1-d2fb-4566-8718-a7842af5f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:20.418159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98383ad1-d2fb-4566-8718-a7842af5f72b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:20.418512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98383ad1-d2fb-4566-8718-a7842af5f72b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:20.418811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98383ad1-d2fb-4566-8718-a7842af5f72b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:20.430982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98383ad1-d2fb-4566-8718-a7842af5f72b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:20.432483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98383ad1-d2fb-4566-8718-a7842af5f72b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-98383ad1-d2fb-4566-8718-a7842af5f72b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:20.454063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:20.455367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:20.456466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:20.456466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:20.456466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:20.456466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:20.456466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 229/465] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:20.458624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a90d43-449c-424b-81cf-76dcc502b927 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:20.459180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a90d43-449c-424b-81cf-76dcc502b927 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:20.459665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a90d43-449c-424b-81cf-76dcc502b927 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:20.467394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86eab0-a3fd-40aa-87af-eabbe46441ab None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:20.468725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 230/466] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:20 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:20.470757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a90d43-449c-424b-81cf-76dcc502b927 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:20.516591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919203d-f11c-4802-b268-219e95bb7c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:20.517254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919203d-f11c-4802-b268-219e95bb7c6d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:20.517789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919203d-f11c-4802-b268-219e95bb7c6d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:20.517789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919203d-f11c-4802-b268-219e95bb7c6d None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:20.518489 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919203d-f11c-4802-b268-219e95bb7c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:20.518825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919203d-f11c-4802-b268-219e95bb7c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:20.520206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919203d-f11c-4802-b268-219e95bb7c6d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:20.538095 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 237/467] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:20.539451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919203d-f11c-4802-b268-219e95bb7c6d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:20.607684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a90d43-449c-424b-81cf-76dcc502b927 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:20.657996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1407af-aff9-4e6e-b4f4-9ab7c9ae670a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:20.658689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1407af-aff9-4e6e-b4f4-9ab7c9ae670a None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:20.659003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1407af-aff9-4e6e-b4f4-9ab7c9ae670a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:20.659275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1407af-aff9-4e6e-b4f4-9ab7c9ae670a None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:20.660006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1407af-aff9-4e6e-b4f4-9ab7c9ae670a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:20.661905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1407af-aff9-4e6e-b4f4-9ab7c9ae670a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:20.670747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919203d-f11c-4802-b268-219e95bb7c6d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:20.692879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 231/468] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:41:20 2022] POST /identity/v3/users => generated 246 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:41:20.695945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbddbb-db04-4e1d-a542-9bd002e24563 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:20.696457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbddbb-db04-4e1d-a542-9bd002e24563 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:20.696949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbddbb-db04-4e1d-a542-9bd002e24563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:20.726120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-4fbbddbb-db04-4e1d-a542-9bd002e24563 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:20.728818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4fbbddbb-db04-4e1d-a542-9bd002e24563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:20.741380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00774870-85e0-4e9b-9e4a-61176e796ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:20.742441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774870-85e0-4e9b-9e4a-61176e796ff4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:20.742995 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774870-85e0-4e9b-9e4a-61176e796ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:20.743489 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774870-85e0-4e9b-9e4a-61176e796ff4 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:20.745600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00774870-85e0-4e9b-9e4a-61176e796ff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:20.746359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00774870-85e0-4e9b-9e4a-61176e796ff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:20.748201 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00774870-85e0-4e9b-9e4a-61176e796ff4 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:20.772321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 238/469] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:20.775363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00774870-85e0-4e9b-9e4a-61176e796ff4 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:20.859152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 232/470] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:20.861040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbddbb-db04-4e1d-a542-9bd002e24563 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:20.954823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00774870-85e0-4e9b-9e4a-61176e796ff4 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:21.006529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a07cf-ba2c-42b7-bd50-1cf2314e4c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:21.007665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a07cf-ba2c-42b7-bd50-1cf2314e4c93 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:21.007993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a07cf-ba2c-42b7-bd50-1cf2314e4c93 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:21.008290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a07cf-ba2c-42b7-bd50-1cf2314e4c93 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:21.009338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a07cf-ba2c-42b7-bd50-1cf2314e4c93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:21.011361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a07cf-ba2c-42b7-bd50-1cf2314e4c93 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:21.031253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbddbb-db04-4e1d-a542-9bd002e24563 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:21.046276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 239/471] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:41:20 2022] POST /identity/v3/users => generated 248 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:41:21.052372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a07cf-ba2c-42b7-bd50-1cf2314e4c93 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:21.092691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbf05b-acd2-4678-add0-375d96dc78a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:21.093865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbf05b-acd2-4678-add0-375d96dc78a4 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:21.094177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbf05b-acd2-4678-add0-375d96dc78a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:21.094519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbf05b-acd2-4678-add0-375d96dc78a4 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:21.095283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbf05b-acd2-4678-add0-375d96dc78a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:21.096930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbf05b-acd2-4678-add0-375d96dc78a4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:21.129197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 233/472] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:41:20 2022] POST /identity/v3/users => generated 248 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:41:21.131211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f092504-935d-4e27-966f-c1c753637a98 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:21.131499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f092504-935d-4e27-966f-c1c753637a98 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:21.131770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f092504-935d-4e27-966f-c1c753637a98 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:21.134779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 234/473] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:21.140571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b5191-7b88-4218-96fc-6d990e24103a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:21.141075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b5191-7b88-4218-96fc-6d990e24103a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:21.142335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b5191-7b88-4218-96fc-6d990e24103a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:21.167915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b50b5191-7b88-4218-96fc-6d990e24103a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:21.169450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b50b5191-7b88-4218-96fc-6d990e24103a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:21.208407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a07cf-ba2c-42b7-bd50-1cf2314e4c93 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:21.252706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e4d2f-9d14-4eba-90f2-43e5cc1974ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:21.253689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e4d2f-9d14-4eba-90f2-43e5cc1974ab None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:21.254071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e4d2f-9d14-4eba-90f2-43e5cc1974ab None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:21.254450 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e4d2f-9d14-4eba-90f2-43e5cc1974ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:21.272218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e4d2f-9d14-4eba-90f2-43e5cc1974ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:21.273908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e4d2f-9d14-4eba-90f2-43e5cc1974ab None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:21.292611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 235/474] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-483e4d2f-9d14-4eba-90f2-43e5cc1974ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:21.311364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:21.312444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:21.313484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:21.313484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:21.313484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:21.313484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:21.315476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 240/475] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:21.322157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b5191-7b88-4218-96fc-6d990e24103a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:21.333003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e4d2f-9d14-4eba-90f2-43e5cc1974ab None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:21.460976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b5191-7b88-4218-96fc-6d990e24103a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:21.484382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e4d2f-9d14-4eba-90f2-43e5cc1974ab None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:21.509867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69048a0-ea13-4187-a59f-bdd3e65d5bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:21.510942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69048a0-ea13-4187-a59f-bdd3e65d5bf6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:21.511423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69048a0-ea13-4187-a59f-bdd3e65d5bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:21.511888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69048a0-ea13-4187-a59f-bdd3e65d5bf6 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:21.512824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69048a0-ea13-4187-a59f-bdd3e65d5bf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:21.513342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69048a0-ea13-4187-a59f-bdd3e65d5bf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:21.514999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69048a0-ea13-4187-a59f-bdd3e65d5bf6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:21.531047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a94a9-8965-4ebd-9360-e99eaeb97135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:21.531602 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 236/476] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:21.532122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a94a9-8965-4ebd-9360-e99eaeb97135 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:21.532648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a94a9-8965-4ebd-9360-e99eaeb97135 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:21.533088 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a94a9-8965-4ebd-9360-e99eaeb97135 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:21.534411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a94a9-8965-4ebd-9360-e99eaeb97135 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:21.536877 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a94a9-8965-4ebd-9360-e99eaeb97135 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:21.539147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69048a0-ea13-4187-a59f-bdd3e65d5bf6 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:21.547686 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.sql.core [None req-eb9a94a9-8965-4ebd-9360-e99eaeb97135 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 13 13:41:21.547686 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 13 13:41:21.547686 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [parameters: {'id': '18f6a94716ca47758558eda071b2a00b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 13 13:41:21.547686 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86363) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-eb9a94a9-8965-4ebd-9360-e99eaeb97135 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application conn.query(q) May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application result.read() May 13 13:41:21.567790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return ref.to_dict() May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:41:21.568744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application next(self.gen) May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application yield resource May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application next(self.gen) May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self.session.rollback() May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self.force_reraise() May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise self.value May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application session.commit() May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 13 13:41:21.569627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._prepare_impl() May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self.session.flush() May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._flush(objects) May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application compat.raise_( May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application flush_context.execute() May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rec.execute(self) May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 13 13:41:21.570467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application _emit_insert_statements( May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application c = connection._execute_20( May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ret = self._execute_context( May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:41:21.571527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application conn.query(q) May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application result.read() May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:41:21.572520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [parameters: {'id': '18f6a94716ca47758558eda071b2a00b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:21.573290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 13 13:41:21.574147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:21.577234 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 241/477] 10.222.0.79 () {60 vars in 1131 bytes} [Fri May 13 13:41:21 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:21.577234 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a94a9-8965-4ebd-9360-e99eaeb97135 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:21.672497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69048a0-ea13-4187-a59f-bdd3e65d5bf6 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:21.694073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a94a9-8965-4ebd-9360-e99eaeb97135 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:21.711419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f448e-af3f-4042-b87d-534b73a5152b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:21.712914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f448e-af3f-4042-b87d-534b73a5152b None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:21.713429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f448e-af3f-4042-b87d-534b73a5152b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:21.714065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f448e-af3f-4042-b87d-534b73a5152b None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:21.715607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f448e-af3f-4042-b87d-534b73a5152b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:21.718466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f448e-af3f-4042-b87d-534b73a5152b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:21.742388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2c2fc-3697-4b30-80fb-5acc8f9eef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:21.744054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2c2fc-3697-4b30-80fb-5acc8f9eef11 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:21.744054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2c2fc-3697-4b30-80fb-5acc8f9eef11 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:21.744377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2c2fc-3697-4b30-80fb-5acc8f9eef11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:21.757199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2c2fc-3697-4b30-80fb-5acc8f9eef11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:21.758638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 237/478] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:41:21 2022] POST /identity/v3/users => generated 249 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-96e2c2fc-3697-4b30-80fb-5acc8f9eef11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:21.766515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:21.767734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:21.768913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 242/479] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:41:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:21.775671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f448e-af3f-4042-b87d-534b73a5152b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:21.982002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f448e-af3f-4042-b87d-534b73a5152b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:22.043442 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273246a-4f68-43e8-a764-9843fda19090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:22.044241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273246a-4f68-43e8-a764-9843fda19090 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:22.044615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273246a-4f68-43e8-a764-9843fda19090 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:22.045058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273246a-4f68-43e8-a764-9843fda19090 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:22.045922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273246a-4f68-43e8-a764-9843fda19090 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:22.046427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273246a-4f68-43e8-a764-9843fda19090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:22.048034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273246a-4f68-43e8-a764-9843fda19090 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:22.060508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 238/480] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:24.244439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65be506-6025-4225-acc2-337344d5bf81 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:24.244935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65be506-6025-4225-acc2-337344d5bf81 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:24.244935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65be506-6025-4225-acc2-337344d5bf81 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:24.246585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 243/481] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:24.253376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eba352-f825-4a53-8637-83dce9c0ac5e None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:24.253860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eba352-f825-4a53-8637-83dce9c0ac5e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:24.254232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eba352-f825-4a53-8637-83dce9c0ac5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:24.288664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-34eba352-f825-4a53-8637-83dce9c0ac5e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:24.289558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a794327-9c11-46d6-afc2-eef23a10f8a6 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:24.289896 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-34eba352-f825-4a53-8637-83dce9c0ac5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:24.290294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a794327-9c11-46d6-afc2-eef23a10f8a6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:24.290840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a794327-9c11-46d6-afc2-eef23a10f8a6 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:24.292739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 244/482] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:24.322965 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b222f8b-4c71-4c9e-90d8-1ff653f7b8f3 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:24.323289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b222f8b-4c71-4c9e-90d8-1ff653f7b8f3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:24.323529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b222f8b-4c71-4c9e-90d8-1ff653f7b8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:24.358281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-4b222f8b-4c71-4c9e-90d8-1ff653f7b8f3 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:24.360202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-4b222f8b-4c71-4c9e-90d8-1ff653f7b8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:24.406448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 239/483] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:24.448078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eba352-f825-4a53-8637-83dce9c0ac5e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:24.448970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 245/484] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:24.517117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b222f8b-4c71-4c9e-90d8-1ff653f7b8f3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:24.600121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eba352-f825-4a53-8637-83dce9c0ac5e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:24.650203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4089553-6385-4f10-a4c6-d12d40546327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:24.651159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4089553-6385-4f10-a4c6-d12d40546327 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:24.651298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4089553-6385-4f10-a4c6-d12d40546327 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:24.651576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4089553-6385-4f10-a4c6-d12d40546327 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:24.671546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4089553-6385-4f10-a4c6-d12d40546327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-f4089553-6385-4f10-a4c6-d12d40546327 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:24.679585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:24.680539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:24.681834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:24.686094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 240/485] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:24.687775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4089553-6385-4f10-a4c6-d12d40546327 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:24.713569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b222f8b-4c71-4c9e-90d8-1ff653f7b8f3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:24.750868 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63c00a-2b41-4ec8-9f1e-b87b433b883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:24.752240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63c00a-2b41-4ec8-9f1e-b87b433b883b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:24.752735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63c00a-2b41-4ec8-9f1e-b87b433b883b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:24.753225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63c00a-2b41-4ec8-9f1e-b87b433b883b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:24.764888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63c00a-2b41-4ec8-9f1e-b87b433b883b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-8e63c00a-2b41-4ec8-9f1e-b87b433b883b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:24.773786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:24.774689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:24.775527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:24.778232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 246/486] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:24.784247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63c00a-2b41-4ec8-9f1e-b87b433b883b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:24.802017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4089553-6385-4f10-a4c6-d12d40546327 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:24.838131 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea579ce-1699-464f-a700-01a069189f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:24.838973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea579ce-1699-464f-a700-01a069189f7c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:24.839159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea579ce-1699-464f-a700-01a069189f7c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:24.839436 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea579ce-1699-464f-a700-01a069189f7c None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:24.840183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea579ce-1699-464f-a700-01a069189f7c None admin] RBAC: Adding query filter params (name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:24.840492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea579ce-1699-464f-a700-01a069189f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:24.841883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea579ce-1699-464f-a700-01a069189f7c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:24.855013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 241/487] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:24.856571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea579ce-1699-464f-a700-01a069189f7c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:24.911148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63c00a-2b41-4ec8-9f1e-b87b433b883b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:24.987569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e6194-ca7b-4c06-a022-fd2a2467f079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:24.988262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e6194-ca7b-4c06-a022-fd2a2467f079 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:24.988546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e6194-ca7b-4c06-a022-fd2a2467f079 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:24.988813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e6194-ca7b-4c06-a022-fd2a2467f079 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:24.989572 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e6194-ca7b-4c06-a022-fd2a2467f079 None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:24.989876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e6194-ca7b-4c06-a022-fd2a2467f079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:24.991332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e6194-ca7b-4c06-a022-fd2a2467f079 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:25.000776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea579ce-1699-464f-a700-01a069189f7c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:25.006509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 247/488] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:25.012064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e6194-ca7b-4c06-a022-fd2a2467f079 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:25.038353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf05ee7-16d3-43cb-8f81-e936ee511940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:25.039101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf05ee7-16d3-43cb-8f81-e936ee511940 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.039380 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf05ee7-16d3-43cb-8f81-e936ee511940 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.039661 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf05ee7-16d3-43cb-8f81-e936ee511940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.052358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf05ee7-16d3-43cb-8f81-e936ee511940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:25.053880 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf05ee7-16d3-43cb-8f81-e936ee511940 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-bdf05ee7-16d3-43cb-8f81-e936ee511940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:25.072996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:25.074503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:25.076086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:25.076086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:25.076086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:25.076086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:25.079620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 242/489] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:25.080593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf05ee7-16d3-43cb-8f81-e936ee511940 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:25.154403 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e6194-ca7b-4c06-a022-fd2a2467f079 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:25.191339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127beba-7093-4ec8-963d-6e24596a7745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:25.192184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127beba-7093-4ec8-963d-6e24596a7745 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.192475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127beba-7093-4ec8-963d-6e24596a7745 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.192851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127beba-7093-4ec8-963d-6e24596a7745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.215457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127beba-7093-4ec8-963d-6e24596a7745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:25.216999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127beba-7093-4ec8-963d-6e24596a7745 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-1127beba-7093-4ec8-963d-6e24596a7745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:25.250957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:25.253399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:25.255253 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:25.255253 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:25.255253 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:25.255253 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:25.255253 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 248/490] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:25.260421 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127beba-7093-4ec8-963d-6e24596a7745 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:25.317939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf05ee7-16d3-43cb-8f81-e936ee511940 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:25.370737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd7ea6-bdc4-49fa-a61f-06336590af25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:25.371900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd7ea6-bdc4-49fa-a61f-06336590af25 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.372407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd7ea6-bdc4-49fa-a61f-06336590af25 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.372908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd7ea6-bdc4-49fa-a61f-06336590af25 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.374328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd7ea6-bdc4-49fa-a61f-06336590af25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:25.374929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd7ea6-bdc4-49fa-a61f-06336590af25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:25.377263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd7ea6-bdc4-49fa-a61f-06336590af25 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:25.388510 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127beba-7093-4ec8-963d-6e24596a7745 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:25.399963 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 243/491] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:25.401371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b435d60-fac7-4a97-87f5-3e4fb44cd697 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.401658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b435d60-fac7-4a97-87f5-3e4fb44cd697 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.401932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b435d60-fac7-4a97-87f5-3e4fb44cd697 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.403652 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 244/492] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:25.411150 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925d003-c097-4d46-b58d-86b673465e98 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.411457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925d003-c097-4d46-b58d-86b673465e98 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.411713 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925d003-c097-4d46-b58d-86b673465e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.430305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b925d003-c097-4d46-b58d-86b673465e98 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:25.432676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b925d003-c097-4d46-b58d-86b673465e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:25.441834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d411c-be08-430b-b8f3-22fcb94b0346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:25.442514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d411c-be08-430b-b8f3-22fcb94b0346 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.442825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d411c-be08-430b-b8f3-22fcb94b0346 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.443192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d411c-be08-430b-b8f3-22fcb94b0346 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.444072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d411c-be08-430b-b8f3-22fcb94b0346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:25.444429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d411c-be08-430b-b8f3-22fcb94b0346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:25.445893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d411c-be08-430b-b8f3-22fcb94b0346 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:25.461699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 249/493] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:25.462585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d411c-be08-430b-b8f3-22fcb94b0346 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:25.530839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 245/494] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:25.531863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925d003-c097-4d46-b58d-86b673465e98 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:25.587254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d411c-be08-430b-b8f3-22fcb94b0346 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:25.624887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11097aca-5f0d-4eff-b47a-5428c5241ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:25.625687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11097aca-5f0d-4eff-b47a-5428c5241ba3 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.625991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11097aca-5f0d-4eff-b47a-5428c5241ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.626263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11097aca-5f0d-4eff-b47a-5428c5241ba3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.649389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11097aca-5f0d-4eff-b47a-5428c5241ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:25.649632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925d003-c097-4d46-b58d-86b673465e98 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:25.650911 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11097aca-5f0d-4eff-b47a-5428c5241ba3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-11097aca-5f0d-4eff-b47a-5428c5241ba3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:25.668577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:25.669800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:25.670700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 250/495] 10.222.0.79 () {56 vars in 1093 bytes} [Fri May 13 13:41:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:25.671552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53030028-1e10-446e-ab41-76228f3bbe97 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.671552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53030028-1e10-446e-ab41-76228f3bbe97 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.671667 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53030028-1e10-446e-ab41-76228f3bbe97 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.672812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 251/496] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:25.673489 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53030028-1e10-446e-ab41-76228f3bbe97 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:25.697094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c1799-931c-41a0-9c28-0f97fab989a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:25.697893 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c1799-931c-41a0-9c28-0f97fab989a7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.698174 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c1799-931c-41a0-9c28-0f97fab989a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.698446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c1799-931c-41a0-9c28-0f97fab989a7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.706672 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c1799-931c-41a0-9c28-0f97fab989a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:25.708756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c1799-931c-41a0-9c28-0f97fab989a7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-a69c1799-931c-41a0-9c28-0f97fab989a7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:25.725794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:25.726792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:25.727694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 246/497] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:25.728750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff2f46-ada3-461a-9f07-62414d8a7d47 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.729022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff2f46-ada3-461a-9f07-62414d8a7d47 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.729302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff2f46-ada3-461a-9f07-62414d8a7d47 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.730594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 247/498] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:25.731315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff2f46-ada3-461a-9f07-62414d8a7d47 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:25.784711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53030028-1e10-446e-ab41-76228f3bbe97 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:25.824381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e077e71-8461-488f-97df-82202f46c55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:25.825220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e077e71-8461-488f-97df-82202f46c55b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.825493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e077e71-8461-488f-97df-82202f46c55b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.825762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e077e71-8461-488f-97df-82202f46c55b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.837901 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e077e71-8461-488f-97df-82202f46c55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:25.838913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff2f46-ada3-461a-9f07-62414d8a7d47 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-8e077e71-8461-488f-97df-82202f46c55b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:25.845594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:25.846522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:25.847371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 252/499] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:25.848406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2773f2-4ad9-41fb-b036-ac5118006bb2 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.849082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2773f2-4ad9-41fb-b036-ac5118006bb2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.849082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2773f2-4ad9-41fb-b036-ac5118006bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.866563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6a2773f2-4ad9-41fb-b036-ac5118006bb2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:25.867799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6a2773f2-4ad9-41fb-b036-ac5118006bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:25.877630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b60e2-584c-40f9-a05b-3e14c13d0440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:25.878287 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b60e2-584c-40f9-a05b-3e14c13d0440 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.878563 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b60e2-584c-40f9-a05b-3e14c13d0440 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.878866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b60e2-584c-40f9-a05b-3e14c13d0440 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.879694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b60e2-584c-40f9-a05b-3e14c13d0440 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:25.879992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b60e2-584c-40f9-a05b-3e14c13d0440 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:25.882426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b60e2-584c-40f9-a05b-3e14c13d0440 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:25.929245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 248/500] 10.222.0.79 () {56 vars in 1133 bytes} [Fri May 13 13:41:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:25.929245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b60e2-584c-40f9-a05b-3e14c13d0440 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:25.948922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 253/501] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:25.950227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d4fdb-b94a-45be-bee9-9caf5d2e4c0e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:25.950511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d4fdb-b94a-45be-bee9-9caf5d2e4c0e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:25.950821 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d4fdb-b94a-45be-bee9-9caf5d2e4c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:25.972557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-4e7d4fdb-b94a-45be-bee9-9caf5d2e4c0e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:25.974385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-4e7d4fdb-b94a-45be-bee9-9caf5d2e4c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:26.057661 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b60e2-584c-40f9-a05b-3e14c13d0440 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:26.058495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 254/502] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:26.059470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d4fdb-b94a-45be-bee9-9caf5d2e4c0e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:26.098141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b2e92-86fb-40ad-a736-892d48c58b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:26.098634 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2e92-86fb-40ad-a736-892d48c58b61 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:26.098944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2e92-86fb-40ad-a736-892d48c58b61 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:26.099394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2e92-86fb-40ad-a736-892d48c58b61 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:26.100245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b2e92-86fb-40ad-a736-892d48c58b61 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:26.100551 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b2e92-86fb-40ad-a736-892d48c58b61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:26.102020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b2e92-86fb-40ad-a736-892d48c58b61 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:26.122545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 249/503] 10.222.0.79 () {56 vars in 1137 bytes} [Fri May 13 13:41:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:26.123454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b2e92-86fb-40ad-a736-892d48c58b61 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:26.176844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d4fdb-b94a-45be-bee9-9caf5d2e4c0e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:26.216438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1b292-6924-42d9-b898-3a69e56a0f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:26.217572 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1b292-6924-42d9-b898-3a69e56a0f7c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:26.218057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1b292-6924-42d9-b898-3a69e56a0f7c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:26.218540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1b292-6924-42d9-b898-3a69e56a0f7c None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:26.219994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1b292-6924-42d9-b898-3a69e56a0f7c None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:26.220398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1b292-6924-42d9-b898-3a69e56a0f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:26.223072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1b292-6924-42d9-b898-3a69e56a0f7c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:26.236555 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 255/504] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:26.237255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b2e92-86fb-40ad-a736-892d48c58b61 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:26.237541 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1b292-6924-42d9-b898-3a69e56a0f7c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:26.278827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13da95c-bc45-45b5-b8c1-da2a8c24e208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:26.279486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13da95c-bc45-45b5-b8c1-da2a8c24e208 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:26.279760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13da95c-bc45-45b5-b8c1-da2a8c24e208 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:26.280029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13da95c-bc45-45b5-b8c1-da2a8c24e208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:26.382214 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13da95c-bc45-45b5-b8c1-da2a8c24e208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:26.384234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13da95c-bc45-45b5-b8c1-da2a8c24e208 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:26.386644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1b292-6924-42d9-b898-3a69e56a0f7c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:26.430210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eacb04-ca01-4642-ba3e-b191031d602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:26.431046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eacb04-ca01-4642-ba3e-b191031d602a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:26.431328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eacb04-ca01-4642-ba3e-b191031d602a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:26.431601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eacb04-ca01-4642-ba3e-b191031d602a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:26.445090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eacb04-ca01-4642-ba3e-b191031d602a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-f0eacb04-ca01-4642-ba3e-b191031d602a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:26.453358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:26.454458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:26.456122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 256/505] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:26.458140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eacb04-ca01-4642-ba3e-b191031d602a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:26.541348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 250/506] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:26.541348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13da95c-bc45-45b5-b8c1-da2a8c24e208 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:26.610421 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eacb04-ca01-4642-ba3e-b191031d602a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:26.660424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ce34b-dd16-404d-a2a7-0c5a70097449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:26.661601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ce34b-dd16-404d-a2a7-0c5a70097449 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:26.662619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ce34b-dd16-404d-a2a7-0c5a70097449 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:26.663113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ce34b-dd16-404d-a2a7-0c5a70097449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:26.681362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ce34b-dd16-404d-a2a7-0c5a70097449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:26.690529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ce34b-dd16-404d-a2a7-0c5a70097449 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-a89ce34b-dd16-404d-a2a7-0c5a70097449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:26.709081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:26.710378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:26.712083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:26.712083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:26.712083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:26.712083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:26.712083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 257/507] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:26.712789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ce34b-dd16-404d-a2a7-0c5a70097449 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:26.716401 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13da95c-bc45-45b5-b8c1-da2a8c24e208 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:26.782623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef53aa-131a-4077-900a-dbea1c931e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:26.783375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef53aa-131a-4077-900a-dbea1c931e2d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:26.783651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef53aa-131a-4077-900a-dbea1c931e2d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:26.783928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef53aa-131a-4077-900a-dbea1c931e2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:26.853942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ce34b-dd16-404d-a2a7-0c5a70097449 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:26.868061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef53aa-131a-4077-900a-dbea1c931e2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:26.870200 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef53aa-131a-4077-900a-dbea1c931e2d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:26.896951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd71bfa-a73b-4c99-aad3-88fe7c5cbbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:26.898227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd71bfa-a73b-4c99-aad3-88fe7c5cbbe9 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:26.898534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd71bfa-a73b-4c99-aad3-88fe7c5cbbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:26.899079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd71bfa-a73b-4c99-aad3-88fe7c5cbbe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:26.913633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd71bfa-a73b-4c99-aad3-88fe7c5cbbe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:26.915105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd71bfa-a73b-4c99-aad3-88fe7c5cbbe9 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-7bd71bfa-a73b-4c99-aad3-88fe7c5cbbe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:26.937216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:26.938144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:26.939064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:26.939064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:26.939064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:26.939064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:26.939064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 258/508] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:26.939731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd71bfa-a73b-4c99-aad3-88fe7c5cbbe9 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:27.000216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 251/509] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:27.000216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef53aa-131a-4077-900a-dbea1c931e2d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:27.079292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd71bfa-a73b-4c99-aad3-88fe7c5cbbe9 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:27.125001 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09017de7-2305-4a8f-95f5-7c8b9941652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:27.125879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09017de7-2305-4a8f-95f5-7c8b9941652d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:27.126335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09017de7-2305-4a8f-95f5-7c8b9941652d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:27.126807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09017de7-2305-4a8f-95f5-7c8b9941652d None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:27.128076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09017de7-2305-4a8f-95f5-7c8b9941652d None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:27.128573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09017de7-2305-4a8f-95f5-7c8b9941652d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:27.131530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09017de7-2305-4a8f-95f5-7c8b9941652d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:27.150350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 259/510] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:27.150550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef53aa-131a-4077-900a-dbea1c931e2d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:27.150660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09017de7-2305-4a8f-95f5-7c8b9941652d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:27.191124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e81574-f753-4167-aa98-de0a0b82e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:27.191931 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e81574-f753-4167-aa98-de0a0b82e3ba None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:27.192212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e81574-f753-4167-aa98-de0a0b82e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:27.192489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e81574-f753-4167-aa98-de0a0b82e3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:27.204551 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e81574-f753-4167-aa98-de0a0b82e3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:27.206286 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e81574-f753-4167-aa98-de0a0b82e3ba None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-07e81574-f753-4167-aa98-de0a0b82e3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:27.226069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:27.227207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:27.228185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:27.228185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:27.228185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:27.228185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:27.228185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 252/511] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:27.228185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e81574-f753-4167-aa98-de0a0b82e3ba None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:27.267098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09017de7-2305-4a8f-95f5-7c8b9941652d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:27.302244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c90029-94c5-4d32-b114-5cb5f2e6674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:27.302935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c90029-94c5-4d32-b114-5cb5f2e6674a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:27.303214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c90029-94c5-4d32-b114-5cb5f2e6674a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:27.303481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c90029-94c5-4d32-b114-5cb5f2e6674a None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:27.304229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c90029-94c5-4d32-b114-5cb5f2e6674a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:27.304548 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c90029-94c5-4d32-b114-5cb5f2e6674a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:27.306446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c90029-94c5-4d32-b114-5cb5f2e6674a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:27.321384 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 260/512] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:27.322253 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c90029-94c5-4d32-b114-5cb5f2e6674a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:27.340909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e81574-f753-4167-aa98-de0a0b82e3ba None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:27.377288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ece43-4a9b-46fd-b168-475a2cc06da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:27.377956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ece43-4a9b-46fd-b168-475a2cc06da7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:27.378244 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ece43-4a9b-46fd-b168-475a2cc06da7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:27.378517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ece43-4a9b-46fd-b168-475a2cc06da7 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:27.379452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ece43-4a9b-46fd-b168-475a2cc06da7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:27.379755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ece43-4a9b-46fd-b168-475a2cc06da7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:27.381675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ece43-4a9b-46fd-b168-475a2cc06da7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:27.393560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 253/513] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:27.394395 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ece43-4a9b-46fd-b168-475a2cc06da7 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:27.439734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c90029-94c5-4d32-b114-5cb5f2e6674a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:27.477780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeda455-cb82-484b-8f35-63f45c2e7f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:27.478487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda455-cb82-484b-8f35-63f45c2e7f6a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:27.478802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda455-cb82-484b-8f35-63f45c2e7f6a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:27.479075 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda455-cb82-484b-8f35-63f45c2e7f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:27.489544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeda455-cb82-484b-8f35-63f45c2e7f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:27.491621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeda455-cb82-484b-8f35-63f45c2e7f6a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-0aeda455-cb82-484b-8f35-63f45c2e7f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:27.511210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:27.512259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:27.513184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:27.513184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:27.513184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:27.513184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:27.513406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ece43-4a9b-46fd-b168-475a2cc06da7 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:27.513480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 261/514] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:27.513480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeda455-cb82-484b-8f35-63f45c2e7f6a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:27.562878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbb861-b0e5-448c-92ac-2f68db131205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:27.564219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbb861-b0e5-448c-92ac-2f68db131205 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:27.564734 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbb861-b0e5-448c-92ac-2f68db131205 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:27.565243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbb861-b0e5-448c-92ac-2f68db131205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:27.586070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbb861-b0e5-448c-92ac-2f68db131205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:27.589055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbb861-b0e5-448c-92ac-2f68db131205 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-10bbb861-b0e5-448c-92ac-2f68db131205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:27.623178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:27.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:27.625011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:27.625011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:27.625011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:27.625011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:27.625655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 254/515] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:27.626824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbb861-b0e5-448c-92ac-2f68db131205 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:27.634523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeda455-cb82-484b-8f35-63f45c2e7f6a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:27.678481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed2a74-e2d8-4ebc-8276-1bfeca0f4032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:27.679365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed2a74-e2d8-4ebc-8276-1bfeca0f4032 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:27.679690 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed2a74-e2d8-4ebc-8276-1bfeca0f4032 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:27.679964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed2a74-e2d8-4ebc-8276-1bfeca0f4032 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:27.680765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed2a74-e2d8-4ebc-8276-1bfeca0f4032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:27.681069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed2a74-e2d8-4ebc-8276-1bfeca0f4032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:27.682787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed2a74-e2d8-4ebc-8276-1bfeca0f4032 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:27.695559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 262/516] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:27.696451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed2a74-e2d8-4ebc-8276-1bfeca0f4032 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:27.782954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbb861-b0e5-448c-92ac-2f68db131205 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:27.822654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b6166-1f6d-4916-aa50-6dcde66b53c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:27.823349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b6166-1f6d-4916-aa50-6dcde66b53c6 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:27.823623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b6166-1f6d-4916-aa50-6dcde66b53c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:27.823908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b6166-1f6d-4916-aa50-6dcde66b53c6 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:27.824594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b6166-1f6d-4916-aa50-6dcde66b53c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:27.826037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b6166-1f6d-4916-aa50-6dcde66b53c6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:27.846496 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 255/517] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:41:27 2022] POST /identity/v3/users => generated 247 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:41:27.847362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b6166-1f6d-4916-aa50-6dcde66b53c6 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:27.850890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed2a74-e2d8-4ebc-8276-1bfeca0f4032 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:27.903045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525753e-4aa0-4d56-ad00-d246bee45648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:27.904287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525753e-4aa0-4d56-ad00-d246bee45648 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:27.904696 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525753e-4aa0-4d56-ad00-d246bee45648 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:27.905008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525753e-4aa0-4d56-ad00-d246bee45648 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:27.920617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525753e-4aa0-4d56-ad00-d246bee45648 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:27.922498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525753e-4aa0-4d56-ad00-d246bee45648 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-4525753e-4aa0-4d56-ad00-d246bee45648 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:27.939859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:27.941568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:27.943559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:27.943559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:27.943559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:27.943559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:27.943559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:27.943559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:27.943559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:27.943559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:27.943559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:41:27.943559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:27.951952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 263/518] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:27.951952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525753e-4aa0-4d56-ad00-d246bee45648 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:28.032157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b6166-1f6d-4916-aa50-6dcde66b53c6 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:28.077039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1b905-f817-41ea-8a22-4ae3dcbd5c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:28.077614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1b905-f817-41ea-8a22-4ae3dcbd5c5a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:28.077939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1b905-f817-41ea-8a22-4ae3dcbd5c5a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:28.078241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1b905-f817-41ea-8a22-4ae3dcbd5c5a None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:28.079033 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1b905-f817-41ea-8a22-4ae3dcbd5c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:28.079335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1b905-f817-41ea-8a22-4ae3dcbd5c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:28.080794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1b905-f817-41ea-8a22-4ae3dcbd5c5a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:28.089102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525753e-4aa0-4d56-ad00-d246bee45648 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:28.098839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 256/519] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:28.100769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1b905-f817-41ea-8a22-4ae3dcbd5c5a None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:28.135820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e538d-262f-4d44-91c1-005acf463f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:28.136488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e538d-262f-4d44-91c1-005acf463f5c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:28.136760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e538d-262f-4d44-91c1-005acf463f5c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:28.137019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e538d-262f-4d44-91c1-005acf463f5c None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:28.137745 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e538d-262f-4d44-91c1-005acf463f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:28.138042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e538d-262f-4d44-91c1-005acf463f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:28.139451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e538d-262f-4d44-91c1-005acf463f5c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:28.154770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 264/520] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:28.155653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e538d-262f-4d44-91c1-005acf463f5c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:28.218772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1b905-f817-41ea-8a22-4ae3dcbd5c5a None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:28.254746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101a230-e2ec-4792-8abf-b8a58fb8e5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:28.255988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101a230-e2ec-4792-8abf-b8a58fb8e5cb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:28.256182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101a230-e2ec-4792-8abf-b8a58fb8e5cb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:28.256541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101a230-e2ec-4792-8abf-b8a58fb8e5cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:28.264695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101a230-e2ec-4792-8abf-b8a58fb8e5cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:28.266313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101a230-e2ec-4792-8abf-b8a58fb8e5cb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:28.269543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e538d-262f-4d44-91c1-005acf463f5c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-6101a230-e2ec-4792-8abf-b8a58fb8e5cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:28.283120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:28.284961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:28.286306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:28.286306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:28.286306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:28.286306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:28.286306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:28.286306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:28.286306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:28.286306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:28.286306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 257/521] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:28.288469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101a230-e2ec-4792-8abf-b8a58fb8e5cb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:28.306544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1260225-947d-43ae-8cc7-e57df0574ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:28.307655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1260225-947d-43ae-8cc7-e57df0574ae0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:28.308122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1260225-947d-43ae-8cc7-e57df0574ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:28.308573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1260225-947d-43ae-8cc7-e57df0574ae0 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:28.309587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1260225-947d-43ae-8cc7-e57df0574ae0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:28.310075 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1260225-947d-43ae-8cc7-e57df0574ae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:28.311735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1260225-947d-43ae-8cc7-e57df0574ae0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:28.352733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 265/522] 10.222.0.79 () {56 vars in 1137 bytes} [Fri May 13 13:41:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:28.353915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1260225-947d-43ae-8cc7-e57df0574ae0 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:28.418799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101a230-e2ec-4792-8abf-b8a58fb8e5cb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:28.457899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390acbf0-4407-4917-8295-db817d6a72a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:28.459286 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390acbf0-4407-4917-8295-db817d6a72a5 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:28.459573 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390acbf0-4407-4917-8295-db817d6a72a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:28.459838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390acbf0-4407-4917-8295-db817d6a72a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:28.466947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390acbf0-4407-4917-8295-db817d6a72a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:28.468416 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390acbf0-4407-4917-8295-db817d6a72a5 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:28.468640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1260225-947d-43ae-8cc7-e57df0574ae0 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-390acbf0-4407-4917-8295-db817d6a72a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:28.484171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:28.485319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:28.486404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:28.486404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:28.486404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:28.486404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:28.486404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:28.486404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:28.486404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:28.486404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:28.492352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 258/523] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:28.492352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390acbf0-4407-4917-8295-db817d6a72a5 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:28.512939 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7d551-a652-47c7-8fec-41b21189c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:28.513947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7d551-a652-47c7-8fec-41b21189c219 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:28.514187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7d551-a652-47c7-8fec-41b21189c219 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:28.514542 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7d551-a652-47c7-8fec-41b21189c219 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:28.522789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7d551-a652-47c7-8fec-41b21189c219 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:28.524359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7d551-a652-47c7-8fec-41b21189c219 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-fed7d551-a652-47c7-8fec-41b21189c219 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:28.540324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:28.541467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:28.542578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:28.542578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:28.542578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:28.542578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:28.542578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:28.542578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:28.542578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:28.542578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:28.542578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:41:28.542578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:28.548331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 266/524] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:28.548331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7d551-a652-47c7-8fec-41b21189c219 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:28.608896 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390acbf0-4407-4917-8295-db817d6a72a5 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:28.647477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70f40a-5c52-4af1-98eb-f6d6b3299efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:28.648311 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70f40a-5c52-4af1-98eb-f6d6b3299efb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:28.648576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70f40a-5c52-4af1-98eb-f6d6b3299efb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:28.648854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70f40a-5c52-4af1-98eb-f6d6b3299efb None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:28.649648 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70f40a-5c52-4af1-98eb-f6d6b3299efb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:28.649954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70f40a-5c52-4af1-98eb-f6d6b3299efb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:28.651484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70f40a-5c52-4af1-98eb-f6d6b3299efb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:28.661882 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7d551-a652-47c7-8fec-41b21189c219 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:28.681279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 259/525] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:28.681279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70f40a-5c52-4af1-98eb-f6d6b3299efb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:28.709686 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0f3a7-d720-4ef0-8025-be45aa76ea9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:28.710348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0f3a7-d720-4ef0-8025-be45aa76ea9b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:28.710635 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0f3a7-d720-4ef0-8025-be45aa76ea9b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:28.710921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0f3a7-d720-4ef0-8025-be45aa76ea9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:28.809702 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0f3a7-d720-4ef0-8025-be45aa76ea9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:28.811212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0f3a7-d720-4ef0-8025-be45aa76ea9b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:28.823307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70f40a-5c52-4af1-98eb-f6d6b3299efb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:28.865645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d39577-a35c-416d-b5c5-7e06c7de7e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:28.866272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d39577-a35c-416d-b5c5-7e06c7de7e32 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:28.866551 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d39577-a35c-416d-b5c5-7e06c7de7e32 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:28.866855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d39577-a35c-416d-b5c5-7e06c7de7e32 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:28.867647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d39577-a35c-416d-b5c5-7e06c7de7e32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:28.867956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d39577-a35c-416d-b5c5-7e06c7de7e32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:28.869808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d39577-a35c-416d-b5c5-7e06c7de7e32 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:28.887683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 260/526] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:28.887683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d39577-a35c-416d-b5c5-7e06c7de7e32 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:28.945235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 267/527] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:28.946478 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0f3a7-d720-4ef0-8025-be45aa76ea9b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:29.021541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d39577-a35c-416d-b5c5-7e06c7de7e32 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:29.060935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ce5e9-9e1a-4fdc-ab92-01687ede3d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:29.062452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ce5e9-9e1a-4fdc-ab92-01687ede3d58 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:29.062775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ce5e9-9e1a-4fdc-ab92-01687ede3d58 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:29.063053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ce5e9-9e1a-4fdc-ab92-01687ede3d58 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:29.064279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ce5e9-9e1a-4fdc-ab92-01687ede3d58 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:29.064600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ce5e9-9e1a-4fdc-ab92-01687ede3d58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:29.066114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ce5e9-9e1a-4fdc-ab92-01687ede3d58 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:29.083147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0f3a7-d720-4ef0-8025-be45aa76ea9b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:29.095277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 261/528] 10.222.0.79 () {56 vars in 1139 bytes} [Fri May 13 13:41:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:29.096588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ce5e9-9e1a-4fdc-ab92-01687ede3d58 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:29.152232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d4808-cc6c-4d28-a076-937a9c2557e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:29.152838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4808-cc6c-4d28-a076-937a9c2557e8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:29.153118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4808-cc6c-4d28-a076-937a9c2557e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:29.153391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4808-cc6c-4d28-a076-937a9c2557e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:29.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d4808-cc6c-4d28-a076-937a9c2557e8 None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=24ad0dd9d265425087be2be8d245b7fc) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:29.154856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d4808-cc6c-4d28-a076-937a9c2557e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:29.156335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d4808-cc6c-4d28-a076-937a9c2557e8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:29.165163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 268/529] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:28 2022] GET /identity/v3/role_assignments?user.id=24ad0dd9d265425087be2be8d245b7fc&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:29.166366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d4808-cc6c-4d28-a076-937a9c2557e8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:29.233928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ce5e9-9e1a-4fdc-ab92-01687ede3d58 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:29.282741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c4746-a8e2-4622-bf65-c7716a659772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:29.284123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c4746-a8e2-4622-bf65-c7716a659772 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:29.284657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c4746-a8e2-4622-bf65-c7716a659772 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:29.285119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c4746-a8e2-4622-bf65-c7716a659772 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:29.286190 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c4746-a8e2-4622-bf65-c7716a659772 None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=cc4f45e859d84ceaa5a01477e83e098e) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:29.287263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c4746-a8e2-4622-bf65-c7716a659772 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:29.289537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c4746-a8e2-4622-bf65-c7716a659772 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:29.294968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d4808-cc6c-4d28-a076-937a9c2557e8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:29.296520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 262/530] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:29 2022] GET /identity/v3/role_assignments?user.id=cc4f45e859d84ceaa5a01477e83e098e&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:29.297593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c4746-a8e2-4622-bf65-c7716a659772 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:29.359395 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff8408-e8cb-4f59-aa30-f0a71412b8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:29.360337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff8408-e8cb-4f59-aa30-f0a71412b8cb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:29.360621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff8408-e8cb-4f59-aa30-f0a71412b8cb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:29.360896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff8408-e8cb-4f59-aa30-f0a71412b8cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:29.389313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff8408-e8cb-4f59-aa30-f0a71412b8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:29.390989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff8408-e8cb-4f59-aa30-f0a71412b8cb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-2bff8408-e8cb-4f59-aa30-f0a71412b8cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:29.428326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:29.429404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:29.430239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:29.430239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:29.430239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:29.430239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:29.434919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 269/531] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:29.436000 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff8408-e8cb-4f59-aa30-f0a71412b8cb None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:29.501415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c4746-a8e2-4622-bf65-c7716a659772 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:29.557913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff8408-e8cb-4f59-aa30-f0a71412b8cb None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:29.561546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d3619-f904-43c7-a9cf-db5676c03f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:29.562595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d3619-f904-43c7-a9cf-db5676c03f05 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:29.563159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d3619-f904-43c7-a9cf-db5676c03f05 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:29.563621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d3619-f904-43c7-a9cf-db5676c03f05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:29.640466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb13d84-aa6c-411a-af16-9adf67fe718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:29.641151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb13d84-aa6c-411a-af16-9adf67fe718b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:29.641433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb13d84-aa6c-411a-af16-9adf67fe718b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:29.641705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb13d84-aa6c-411a-af16-9adf67fe718b None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:29.642444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb13d84-aa6c-411a-af16-9adf67fe718b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:29.642778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb13d84-aa6c-411a-af16-9adf67fe718b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:29.644197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb13d84-aa6c-411a-af16-9adf67fe718b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:29.684929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 270/532] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:29.690223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb13d84-aa6c-411a-af16-9adf67fe718b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:29.731309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d3619-f904-43c7-a9cf-db5676c03f05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:29.732968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d3619-f904-43c7-a9cf-db5676c03f05 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:29.867098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 263/533] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:29.868532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d3619-f904-43c7-a9cf-db5676c03f05 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:29.988041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb13d84-aa6c-411a-af16-9adf67fe718b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:30.017850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d3619-f904-43c7-a9cf-db5676c03f05 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:30.066143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2809d-86dc-41db-9752-4ebd416e5ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:30.067139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2809d-86dc-41db-9752-4ebd416e5ad2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:30.067433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2809d-86dc-41db-9752-4ebd416e5ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:30.068347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2809d-86dc-41db-9752-4ebd416e5ad2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:30.082996 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2809d-86dc-41db-9752-4ebd416e5ad2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:30.085051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2809d-86dc-41db-9752-4ebd416e5ad2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:30.088087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d32fd3-4eb1-4206-90a3-724be0ef1dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:30.088941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d32fd3-4eb1-4206-90a3-724be0ef1dbc None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:30.089303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d32fd3-4eb1-4206-90a3-724be0ef1dbc None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:30.089615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d32fd3-4eb1-4206-90a3-724be0ef1dbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:30.115153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d32fd3-4eb1-4206-90a3-724be0ef1dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:30.117158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d32fd3-4eb1-4206-90a3-724be0ef1dbc None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-68b2809d-86dc-41db-9752-4ebd416e5ad2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:30.120812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:30.122923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:30.125003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:30.125003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:30.125003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:30.125003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:30.125003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:30.125003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:30.125003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:30.125003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:30.125003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 271/534] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:30.129151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2809d-86dc-41db-9752-4ebd416e5ad2 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-46d32fd3-4eb1-4206-90a3-724be0ef1dbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:30.151106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:30.153312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:30.155514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:30.155514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:30.155514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:30.155514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:30.158740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 264/535] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:30.159472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d32fd3-4eb1-4206-90a3-724be0ef1dbc None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:30.291566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d32fd3-4eb1-4206-90a3-724be0ef1dbc None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:30.303302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2809d-86dc-41db-9752-4ebd416e5ad2 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:30.331670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca74ed-ff8a-4201-aee2-bdab524ce260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:30.332384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca74ed-ff8a-4201-aee2-bdab524ce260 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:30.332662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca74ed-ff8a-4201-aee2-bdab524ce260 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:30.332937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca74ed-ff8a-4201-aee2-bdab524ce260 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:30.334403 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca74ed-ff8a-4201-aee2-bdab524ce260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:30.334779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca74ed-ff8a-4201-aee2-bdab524ce260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:30.336344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca74ed-ff8a-4201-aee2-bdab524ce260 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:30.355019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 265/536] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:30.357491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca74ed-ff8a-4201-aee2-bdab524ce260 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:30.385027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2be8c-d972-4bfe-a732-dbb69da8d161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:30.385734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2be8c-d972-4bfe-a732-dbb69da8d161 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:30.386018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2be8c-d972-4bfe-a732-dbb69da8d161 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:30.386296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2be8c-d972-4bfe-a732-dbb69da8d161 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:30.387181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2be8c-d972-4bfe-a732-dbb69da8d161 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:30.387494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2be8c-d972-4bfe-a732-dbb69da8d161 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:30.389315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2be8c-d972-4bfe-a732-dbb69da8d161 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:30.423346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 272/537] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:30.429691 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2be8c-d972-4bfe-a732-dbb69da8d161 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:30.516885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca74ed-ff8a-4201-aee2-bdab524ce260 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:30.558310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2be8c-d972-4bfe-a732-dbb69da8d161 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:30.567570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e2678-07c7-4bfd-9b09-014bb5688848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:30.568912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e2678-07c7-4bfd-9b09-014bb5688848 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:30.569351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e2678-07c7-4bfd-9b09-014bb5688848 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:30.569737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e2678-07c7-4bfd-9b09-014bb5688848 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:30.581891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e2678-07c7-4bfd-9b09-014bb5688848 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:30.584898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e2678-07c7-4bfd-9b09-014bb5688848 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-470e2678-07c7-4bfd-9b09-014bb5688848 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:30.608788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:30.611122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:30.612441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:30.612441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:30.612441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:30.612441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:30.612441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:30.612441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:30.612441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:30.612441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:30.618084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 266/538] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:30.619546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e2678-07c7-4bfd-9b09-014bb5688848 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:30.628803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf6ed6-342a-45d1-bc56-73cb07884162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:30.629874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf6ed6-342a-45d1-bc56-73cb07884162 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:30.630306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf6ed6-342a-45d1-bc56-73cb07884162 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:30.630833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf6ed6-342a-45d1-bc56-73cb07884162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:30.632435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf6ed6-342a-45d1-bc56-73cb07884162 None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=e03fa33a68ff4567bc25c9611cd9ae8b) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:30.632978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf6ed6-342a-45d1-bc56-73cb07884162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:30.635101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf6ed6-342a-45d1-bc56-73cb07884162 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:30.651903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 273/539] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:30 2022] GET /identity/v3/role_assignments?user.id=e03fa33a68ff4567bc25c9611cd9ae8b&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:30.739188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410252e-4ce1-439e-a7b0-f56c152e0822 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:30.739714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410252e-4ce1-439e-a7b0-f56c152e0822 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:30.740190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410252e-4ce1-439e-a7b0-f56c152e0822 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:30.750483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 274/540] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:30 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:41:30.757246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479e534-b2f3-468f-b1fe-a2b9bd55d3df None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:30.757246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479e534-b2f3-468f-b1fe-a2b9bd55d3df None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:30.757246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479e534-b2f3-468f-b1fe-a2b9bd55d3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:30.807391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e2678-07c7-4bfd-9b09-014bb5688848 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:30.811582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-3479e534-b2f3-468f-b1fe-a2b9bd55d3df None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:30.812815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-3479e534-b2f3-468f-b1fe-a2b9bd55d3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:30.897956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dad9e-df00-4a2f-b929-8fdbd7f4a625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:30.898640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dad9e-df00-4a2f-b929-8fdbd7f4a625 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:30.898959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dad9e-df00-4a2f-b929-8fdbd7f4a625 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:30.899235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dad9e-df00-4a2f-b929-8fdbd7f4a625 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:30.900056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dad9e-df00-4a2f-b929-8fdbd7f4a625 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:30.900370 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dad9e-df00-4a2f-b929-8fdbd7f4a625 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:30.901933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dad9e-df00-4a2f-b929-8fdbd7f4a625 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:30.948248 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 267/541] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:30.978047 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dad9e-df00-4a2f-b929-8fdbd7f4a625 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:30.981659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 275/542] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:31.013968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479e534-b2f3-468f-b1fe-a2b9bd55d3df None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:31.214586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479e534-b2f3-468f-b1fe-a2b9bd55d3df None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:31.222649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dad9e-df00-4a2f-b929-8fdbd7f4a625 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:31.264887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e2330-c671-4c91-9921-b32998e994d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:31.265635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e2330-c671-4c91-9921-b32998e994d4 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:31.266014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e2330-c671-4c91-9921-b32998e994d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:31.266297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e2330-c671-4c91-9921-b32998e994d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:31.267332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e2330-c671-4c91-9921-b32998e994d4 None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=8f784cc461ab48ad92140680758a3704) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:31.267877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e2330-c671-4c91-9921-b32998e994d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:31.269751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e2330-c671-4c91-9921-b32998e994d4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:31.277139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 268/543] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:30 2022] GET /identity/v3/role_assignments?user.id=8f784cc461ab48ad92140680758a3704&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:31.291392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8985a-fe23-4130-93ac-46f348f8ed8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:31.292868 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8985a-fe23-4130-93ac-46f348f8ed8c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:31.293355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8985a-fe23-4130-93ac-46f348f8ed8c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:31.293645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8985a-fe23-4130-93ac-46f348f8ed8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:31.312910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8985a-fe23-4130-93ac-46f348f8ed8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-48c8985a-fe23-4130-93ac-46f348f8ed8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:31.327619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:31.328785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:31.329628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 276/544] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:31.337950 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e2330-c671-4c91-9921-b32998e994d4 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:31.469186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e2330-c671-4c91-9921-b32998e994d4 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:31.512533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eb3aa-d1ca-434c-ad3c-6d0fd0e0d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:31.513216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eb3aa-d1ca-434c-ad3c-6d0fd0e0d861 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:31.513499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eb3aa-d1ca-434c-ad3c-6d0fd0e0d861 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:31.513779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eb3aa-d1ca-434c-ad3c-6d0fd0e0d861 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:31.514528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eb3aa-d1ca-434c-ad3c-6d0fd0e0d861 None admin] RBAC: Adding query filter params (name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:31.514964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eb3aa-d1ca-434c-ad3c-6d0fd0e0d861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:31.516408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eb3aa-d1ca-434c-ad3c-6d0fd0e0d861 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:31.529018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 269/545] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:31.535743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8985a-fe23-4130-93ac-46f348f8ed8c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:31.657287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8985a-fe23-4130-93ac-46f348f8ed8c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:31.695079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37936918-bc27-464e-9890-6509aed63d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:31.695880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37936918-bc27-464e-9890-6509aed63d5c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:31.696164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37936918-bc27-464e-9890-6509aed63d5c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:31.696439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37936918-bc27-464e-9890-6509aed63d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:31.713335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37936918-bc27-464e-9890-6509aed63d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:31.714844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37936918-bc27-464e-9890-6509aed63d5c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-37936918-bc27-464e-9890-6509aed63d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:31.739369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:31.741601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:31.744269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:31.744269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:31.744269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:31.744269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:31.747119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 277/546] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:31.750851 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eb3aa-d1ca-434c-ad3c-6d0fd0e0d861 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:31.902056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eb3aa-d1ca-434c-ad3c-6d0fd0e0d861 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:31.980904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e73884-1f0b-4be0-a3ec-d48f7460f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:31.982095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e73884-1f0b-4be0-a3ec-d48f7460f9be None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:31.982624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e73884-1f0b-4be0-a3ec-d48f7460f9be None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:31.983185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e73884-1f0b-4be0-a3ec-d48f7460f9be None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:31.984704 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e73884-1f0b-4be0-a3ec-d48f7460f9be None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:31.985298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e73884-1f0b-4be0-a3ec-d48f7460f9be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:31.988152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e73884-1f0b-4be0-a3ec-d48f7460f9be None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:32.025119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 270/547] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:32.030287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37936918-bc27-464e-9890-6509aed63d5c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:32.176543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37936918-bc27-464e-9890-6509aed63d5c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:32.230586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c9517-f2a7-40e9-afa7-a4efe3163655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:32.231732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c9517-f2a7-40e9-afa7-a4efe3163655 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:32.232098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c9517-f2a7-40e9-afa7-a4efe3163655 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:32.232385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c9517-f2a7-40e9-afa7-a4efe3163655 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:32.251185 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c9517-f2a7-40e9-afa7-a4efe3163655 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:32.252957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c9517-f2a7-40e9-afa7-a4efe3163655 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-894c9517-f2a7-40e9-afa7-a4efe3163655 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:32.269989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:32.271971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:32.273734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:32.273734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:32.273734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:32.273734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:32.273734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:32.273734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:32.273734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:32.273734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:32.273734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:41:32.273734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:32.277964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 278/548] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:41:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:32.287226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e73884-1f0b-4be0-a3ec-d48f7460f9be None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:32.465053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e73884-1f0b-4be0-a3ec-d48f7460f9be None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:32.533862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b5900-dc02-4942-9c22-690f5a49ddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:32.535039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b5900-dc02-4942-9c22-690f5a49ddb1 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:32.535438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b5900-dc02-4942-9c22-690f5a49ddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:32.535935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b5900-dc02-4942-9c22-690f5a49ddb1 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:32.538323 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b5900-dc02-4942-9c22-690f5a49ddb1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:32.538927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b5900-dc02-4942-9c22-690f5a49ddb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:32.541284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b5900-dc02-4942-9c22-690f5a49ddb1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:32.577665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 271/549] 10.222.0.79 () {56 vars in 1135 bytes} [Fri May 13 13:41:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:32.586296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c9517-f2a7-40e9-afa7-a4efe3163655 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:32.811156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ff044-c947-4160-8086-35a3dd29deef None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:32.811640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ff044-c947-4160-8086-35a3dd29deef None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:32.811640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ff044-c947-4160-8086-35a3dd29deef None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:32.813389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 272/550] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:41:32.821910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5bbdc-1212-4a12-82ea-02734c2d98f7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:32.822337 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5bbdc-1212-4a12-82ea-02734c2d98f7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:32.822582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5bbdc-1212-4a12-82ea-02734c2d98f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:32.851077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-01d5bbdc-1212-4a12-82ea-02734c2d98f7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:32.852341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-01d5bbdc-1212-4a12-82ea-02734c2d98f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:32.857761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c9517-f2a7-40e9-afa7-a4efe3163655 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:32.924472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe3cb1-9739-4f1c-b234-8f541b0427cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:32.925578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe3cb1-9739-4f1c-b234-8f541b0427cb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:32.926072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe3cb1-9739-4f1c-b234-8f541b0427cb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:32.926560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe3cb1-9739-4f1c-b234-8f541b0427cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:32.944811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 273/551] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:32.946206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dbc91-05fa-4054-a178-722728b5f27e None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:32.946498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dbc91-05fa-4054-a178-722728b5f27e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:32.946765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dbc91-05fa-4054-a178-722728b5f27e None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:32.952680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 274/552] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:32.954293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac93ec7-a9ea-45ed-9d88-e11e5704b3a8 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:32.954665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac93ec7-a9ea-45ed-9d88-e11e5704b3a8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:32.955059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac93ec7-a9ea-45ed-9d88-e11e5704b3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:32.978963 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7ac93ec7-a9ea-45ed-9d88-e11e5704b3a8 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:32.978963 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7ac93ec7-a9ea-45ed-9d88-e11e5704b3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:33.048541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 275/553] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:33.049438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac93ec7-a9ea-45ed-9d88-e11e5704b3a8 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:33.055577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe3cb1-9739-4f1c-b234-8f541b0427cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:33.058279 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe3cb1-9739-4f1c-b234-8f541b0427cb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:33.185903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac93ec7-a9ea-45ed-9d88-e11e5704b3a8 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:33.215309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 279/554] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:33.216209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe3cb1-9739-4f1c-b234-8f541b0427cb None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:33.224140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e858fdc-effe-4716-9287-56961add50c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:33.225018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e858fdc-effe-4716-9287-56961add50c3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:33.225298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e858fdc-effe-4716-9287-56961add50c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:33.225572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e858fdc-effe-4716-9287-56961add50c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:33.237335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e858fdc-effe-4716-9287-56961add50c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-0e858fdc-effe-4716-9287-56961add50c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:33.245533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:33.246477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:33.247304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 276/555] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:33.248575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e858fdc-effe-4716-9287-56961add50c3 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:33.336171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe3cb1-9739-4f1c-b234-8f541b0427cb None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:33.361521 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e858fdc-effe-4716-9287-56961add50c3 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:33.381180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa143e2-c8f1-4d8f-a813-d87ae4a2dd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:33.382082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa143e2-c8f1-4d8f-a813-d87ae4a2dd86 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:33.382467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa143e2-c8f1-4d8f-a813-d87ae4a2dd86 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:33.382768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa143e2-c8f1-4d8f-a813-d87ae4a2dd86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:33.399289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa143e2-c8f1-4d8f-a813-d87ae4a2dd86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-9aa143e2-c8f1-4d8f-a813-d87ae4a2dd86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:33.408114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:33.411117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:33.413285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:33.414873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7a6aa-7b53-4b37-a17a-10d3d51acf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:33.415124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 280/556] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:33.415282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7a6aa-7b53-4b37-a17a-10d3d51acf7e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:33.415282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7a6aa-7b53-4b37-a17a-10d3d51acf7e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:33.415282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7a6aa-7b53-4b37-a17a-10d3d51acf7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:33.415802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa143e2-c8f1-4d8f-a813-d87ae4a2dd86 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:33.429253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7a6aa-7b53-4b37-a17a-10d3d51acf7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:33.431057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7a6aa-7b53-4b37-a17a-10d3d51acf7e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-f4e7a6aa-7b53-4b37-a17a-10d3d51acf7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:33.452391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:33.454441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:33.456819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:33.456819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:33.456819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:33.456819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:33.462806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 277/557] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:33.462806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7a6aa-7b53-4b37-a17a-10d3d51acf7e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:33.535911 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa143e2-c8f1-4d8f-a813-d87ae4a2dd86 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:33.578277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c3ddf-e0f9-4d76-9430-5444da35101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:33.579257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c3ddf-e0f9-4d76-9430-5444da35101d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:33.579503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c3ddf-e0f9-4d76-9430-5444da35101d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:33.579663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7a6aa-7b53-4b37-a17a-10d3d51acf7e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:33.580151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c3ddf-e0f9-4d76-9430-5444da35101d None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:33.580860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c3ddf-e0f9-4d76-9430-5444da35101d None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:33.581172 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c3ddf-e0f9-4d76-9430-5444da35101d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:33.582807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c3ddf-e0f9-4d76-9430-5444da35101d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:33.597409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 281/558] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:33.598338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c3ddf-e0f9-4d76-9430-5444da35101d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:33.619808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a51a06-e675-4446-8e35-76fb004a5754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:33.620519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a51a06-e675-4446-8e35-76fb004a5754 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:33.620810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a51a06-e675-4446-8e35-76fb004a5754 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:33.621091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a51a06-e675-4446-8e35-76fb004a5754 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:33.621959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a51a06-e675-4446-8e35-76fb004a5754 None admin] RBAC: Adding query filter params (name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:33.622275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a51a06-e675-4446-8e35-76fb004a5754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:33.623759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a51a06-e675-4446-8e35-76fb004a5754 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:33.638215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 278/559] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:33.639202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a51a06-e675-4446-8e35-76fb004a5754 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:33.737287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c3ddf-e0f9-4d76-9430-5444da35101d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:33.763967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a51a06-e675-4446-8e35-76fb004a5754 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:33.779888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae90bb-ed47-4978-9e3b-61454469ded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:33.780752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae90bb-ed47-4978-9e3b-61454469ded5 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:33.781023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae90bb-ed47-4978-9e3b-61454469ded5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:33.781285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae90bb-ed47-4978-9e3b-61454469ded5 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:33.782127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae90bb-ed47-4978-9e3b-61454469ded5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:33.782592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae90bb-ed47-4978-9e3b-61454469ded5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:33.784157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae90bb-ed47-4978-9e3b-61454469ded5 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:33.799342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 282/560] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:33.800343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae90bb-ed47-4978-9e3b-61454469ded5 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:33.818319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fe315-cdfe-452d-b34f-f6ad7d1f2605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:33.819917 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fe315-cdfe-452d-b34f-f6ad7d1f2605 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:33.820241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fe315-cdfe-452d-b34f-f6ad7d1f2605 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:33.820481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fe315-cdfe-452d-b34f-f6ad7d1f2605 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:33.836582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fe315-cdfe-452d-b34f-f6ad7d1f2605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:33.839417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fe315-cdfe-452d-b34f-f6ad7d1f2605 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-184fe315-cdfe-452d-b34f-f6ad7d1f2605 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:33.869082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:33.871176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:33.873193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:33.873193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:33.873193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:33.873193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:33.873193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 279/561] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:33.873193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fe315-cdfe-452d-b34f-f6ad7d1f2605 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:33.935043 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae90bb-ed47-4978-9e3b-61454469ded5 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:33.986967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09a8c8-85d0-4a00-9a50-59a22fd2357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:33.987547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09a8c8-85d0-4a00-9a50-59a22fd2357a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:33.987840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09a8c8-85d0-4a00-9a50-59a22fd2357a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:33.988132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09a8c8-85d0-4a00-9a50-59a22fd2357a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.001455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09a8c8-85d0-4a00-9a50-59a22fd2357a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:34.004266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09a8c8-85d0-4a00-9a50-59a22fd2357a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-9b09a8c8-85d0-4a00-9a50-59a22fd2357a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:34.038206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:34.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:34.042820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:34.042820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:34.042820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:34.042820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:34.042820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 283/562] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:34.042820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09a8c8-85d0-4a00-9a50-59a22fd2357a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:34.052032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fe315-cdfe-452d-b34f-f6ad7d1f2605 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:34.097378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a88610-420f-49e7-adf5-49882f35b541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:34.098186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a88610-420f-49e7-adf5-49882f35b541 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.098539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a88610-420f-49e7-adf5-49882f35b541 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.098752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a88610-420f-49e7-adf5-49882f35b541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.106957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a88610-420f-49e7-adf5-49882f35b541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:34.108440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a88610-420f-49e7-adf5-49882f35b541 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-77a88610-420f-49e7-adf5-49882f35b541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:34.128922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:34.130025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:34.131179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:34.131179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:34.131179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:34.131179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:34.131179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:34.131179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:34.131179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:34.131179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:34.135458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 280/563] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:34.136812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a88610-420f-49e7-adf5-49882f35b541 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:34.187372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09a8c8-85d0-4a00-9a50-59a22fd2357a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:34.234187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730acb56-c847-4eba-a919-2a596663e9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:34.235533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730acb56-c847-4eba-a919-2a596663e9cd None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.236099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730acb56-c847-4eba-a919-2a596663e9cd None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.236603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730acb56-c847-4eba-a919-2a596663e9cd None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.238103 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730acb56-c847-4eba-a919-2a596663e9cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:34.238593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730acb56-c847-4eba-a919-2a596663e9cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:34.241771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730acb56-c847-4eba-a919-2a596663e9cd None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:34.266751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 284/564] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:34.266751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730acb56-c847-4eba-a919-2a596663e9cd None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:34.282543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a88610-420f-49e7-adf5-49882f35b541 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:34.329408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44fe87-734f-4529-95d6-d8639ca2aaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:34.330532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44fe87-734f-4529-95d6-d8639ca2aaa2 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.331049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44fe87-734f-4529-95d6-d8639ca2aaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.331531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44fe87-734f-4529-95d6-d8639ca2aaa2 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.332882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44fe87-734f-4529-95d6-d8639ca2aaa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:34.333968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44fe87-734f-4529-95d6-d8639ca2aaa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:34.336554 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44fe87-734f-4529-95d6-d8639ca2aaa2 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:34.358630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 281/565] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:34.361238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebcb8d-d434-4c51-812f-070cb11ce471 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.361719 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebcb8d-d434-4c51-812f-070cb11ce471 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.362170 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebcb8d-d434-4c51-812f-070cb11ce471 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.364308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 282/566] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:34.366196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebcb8d-d434-4c51-812f-070cb11ce471 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:34.394590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730acb56-c847-4eba-a919-2a596663e9cd None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:34.441044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520a827-c4e7-4131-b0d2-01acee8817a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:34.441726 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520a827-c4e7-4131-b0d2-01acee8817a4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.442048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520a827-c4e7-4131-b0d2-01acee8817a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.442401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520a827-c4e7-4131-b0d2-01acee8817a4 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.443327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520a827-c4e7-4131-b0d2-01acee8817a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:34.443642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520a827-c4e7-4131-b0d2-01acee8817a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:34.445182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520a827-c4e7-4131-b0d2-01acee8817a4 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:34.469630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 285/567] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:34.469630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520a827-c4e7-4131-b0d2-01acee8817a4 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:34.508238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebcb8d-d434-4c51-812f-070cb11ce471 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:34.558196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc883ac-0630-4ddf-a1f1-18452feef583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:34.559314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc883ac-0630-4ddf-a1f1-18452feef583 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.559762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc883ac-0630-4ddf-a1f1-18452feef583 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.560213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc883ac-0630-4ddf-a1f1-18452feef583 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.573525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc883ac-0630-4ddf-a1f1-18452feef583 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:34.575552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc883ac-0630-4ddf-a1f1-18452feef583 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-dfc883ac-0630-4ddf-a1f1-18452feef583 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:34.600320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:34.602278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:34.603257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 283/568] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:34.604362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feada7b-2ce8-40f2-958a-52f5e9db2716 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.604712 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feada7b-2ce8-40f2-958a-52f5e9db2716 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.605271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feada7b-2ce8-40f2-958a-52f5e9db2716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.626736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-6feada7b-2ce8-40f2-958a-52f5e9db2716 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:34.628489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6feada7b-2ce8-40f2-958a-52f5e9db2716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:34.634433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520a827-c4e7-4131-b0d2-01acee8817a4 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:34.679143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467957a3-df54-42d7-a31f-1006f01c476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:34.680143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467957a3-df54-42d7-a31f-1006f01c476a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.680486 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467957a3-df54-42d7-a31f-1006f01c476a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.680829 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467957a3-df54-42d7-a31f-1006f01c476a None admin] PATH_INFO: `/v3/users/nova` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.691083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467957a3-df54-42d7-a31f-1006f01c476a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:34.692808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467957a3-df54-42d7-a31f-1006f01c476a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-467957a3-df54-42d7-a31f-1006f01c476a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:34.709024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:34.710295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 286/569] 10.222.0.79 () {56 vars in 1093 bytes} [Fri May 13 13:41:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:34.712090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467957a3-df54-42d7-a31f-1006f01c476a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:34.729262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 284/570] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:34.730144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feada7b-2ce8-40f2-958a-52f5e9db2716 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:34.866393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467957a3-df54-42d7-a31f-1006f01c476a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:34.872819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feada7b-2ce8-40f2-958a-52f5e9db2716 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:34.917761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f21074-789d-4410-8cf8-28323946b9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:34.918507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f21074-789d-4410-8cf8-28323946b9dd None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.918898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f21074-789d-4410-8cf8-28323946b9dd None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.919247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f21074-789d-4410-8cf8-28323946b9dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.920175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f21074-789d-4410-8cf8-28323946b9dd None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=4139cfe906bd4c7d94226ab589577627) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:34.920597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f21074-789d-4410-8cf8-28323946b9dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:34.922223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f21074-789d-4410-8cf8-28323946b9dd None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:34.924727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607f55e-2679-4fe1-b6c5-f2c67ea3b3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:34.925534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f55e-2679-4fe1-b6c5-f2c67ea3b3f7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.925966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f55e-2679-4fe1-b6c5-f2c67ea3b3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.926333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f55e-2679-4fe1-b6c5-f2c67ea3b3f7 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.928322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607f55e-2679-4fe1-b6c5-f2c67ea3b3f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:34.928838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607f55e-2679-4fe1-b6c5-f2c67ea3b3f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:34.930451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607f55e-2679-4fe1-b6c5-f2c67ea3b3f7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:34.930795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 287/571] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:34 2022] GET /identity/v3/role_assignments?user.id=4139cfe906bd4c7d94226ab589577627&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:34.932271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696a1cf-dfb9-41c5-bf55-34961eebc465 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:34.932705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696a1cf-dfb9-41c5-bf55-34961eebc465 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:34.933111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696a1cf-dfb9-41c5-bf55-34961eebc465 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:34.940125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 288/572] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:34.940125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696a1cf-dfb9-41c5-bf55-34961eebc465 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:34.967525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 285/573] 10.222.0.79 () {56 vars in 1137 bytes} [Fri May 13 13:41:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:34.972992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607f55e-2679-4fe1-b6c5-f2c67ea3b3f7 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:35.106626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696a1cf-dfb9-41c5-bf55-34961eebc465 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:35.144649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607f55e-2679-4fe1-b6c5-f2c67ea3b3f7 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:35.162899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc889e-1b51-4188-805d-3da639ef2332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:35.164102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc889e-1b51-4188-805d-3da639ef2332 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:35.164538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc889e-1b51-4188-805d-3da639ef2332 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:35.164990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc889e-1b51-4188-805d-3da639ef2332 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:35.166153 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc889e-1b51-4188-805d-3da639ef2332 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:35.166688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc889e-1b51-4188-805d-3da639ef2332 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:35.168616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc889e-1b51-4188-805d-3da639ef2332 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:35.194293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 289/574] 10.222.0.79 () {56 vars in 1133 bytes} [Fri May 13 13:41:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:35.198196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb344ffb-35ba-4e88-ae6d-1084a89c5680 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:35.198196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb344ffb-35ba-4e88-ae6d-1084a89c5680 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:35.198196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb344ffb-35ba-4e88-ae6d-1084a89c5680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:35.227426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d49d6c-bc4b-451d-a7fd-d40cb05855ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:35.228419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d49d6c-bc4b-451d-a7fd-d40cb05855ae None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:35.228798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d49d6c-bc4b-451d-a7fd-d40cb05855ae None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:35.228966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-eb344ffb-35ba-4e88-ae6d-1084a89c5680 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:35.229362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d49d6c-bc4b-451d-a7fd-d40cb05855ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:35.231005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-eb344ffb-35ba-4e88-ae6d-1084a89c5680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:35.243453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d49d6c-bc4b-451d-a7fd-d40cb05855ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-52d49d6c-bc4b-451d-a7fd-d40cb05855ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:35.257879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:35.259568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:35.261001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 286/575] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:35.262349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d49d6c-bc4b-451d-a7fd-d40cb05855ae None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:35.358980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 290/576] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:35.360612 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb344ffb-35ba-4e88-ae6d-1084a89c5680 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:35.465332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d49d6c-bc4b-451d-a7fd-d40cb05855ae None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:35.529103 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd9b0a-e932-4041-a8a2-ad83a7a4d5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:35.530350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd9b0a-e932-4041-a8a2-ad83a7a4d5e3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:35.530832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd9b0a-e932-4041-a8a2-ad83a7a4d5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:35.531265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd9b0a-e932-4041-a8a2-ad83a7a4d5e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:35.548706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd9b0a-e932-4041-a8a2-ad83a7a4d5e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:35.551139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd9b0a-e932-4041-a8a2-ad83a7a4d5e3 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:35.577653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb344ffb-35ba-4e88-ae6d-1084a89c5680 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-c4fd9b0a-e932-4041-a8a2-ad83a7a4d5e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:35.585058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:35.586700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:35.595932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:35.595932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:35.595932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:35.595932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:35.595932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 287/577] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:35.595932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd9b0a-e932-4041-a8a2-ad83a7a4d5e3 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:35.647508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae2ae-97f7-4e68-b482-1aaaa11c519f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:35.649357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae2ae-97f7-4e68-b482-1aaaa11c519f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:35.649984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae2ae-97f7-4e68-b482-1aaaa11c519f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:35.650638 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae2ae-97f7-4e68-b482-1aaaa11c519f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:35.672229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efae2ae-97f7-4e68-b482-1aaaa11c519f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:35.675349 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efae2ae-97f7-4e68-b482-1aaaa11c519f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-7efae2ae-97f7-4e68-b482-1aaaa11c519f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:35.706131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:35.707661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:35.709845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:35.709845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:35.709845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:35.709845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:35.715700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 291/578] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:35.717334 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae2ae-97f7-4e68-b482-1aaaa11c519f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:35.776814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd9b0a-e932-4041-a8a2-ad83a7a4d5e3 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:35.837040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5384cd7-e7dd-4139-a3b2-241c32e2557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:35.837843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5384cd7-e7dd-4139-a3b2-241c32e2557f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:35.838184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5384cd7-e7dd-4139-a3b2-241c32e2557f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:35.838519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5384cd7-e7dd-4139-a3b2-241c32e2557f None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:35.839417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5384cd7-e7dd-4139-a3b2-241c32e2557f None admin] RBAC: Adding query filter params (name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:35.839787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5384cd7-e7dd-4139-a3b2-241c32e2557f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:35.841747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5384cd7-e7dd-4139-a3b2-241c32e2557f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:35.857654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 288/579] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:35.857654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5384cd7-e7dd-4139-a3b2-241c32e2557f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:35.890993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae2ae-97f7-4e68-b482-1aaaa11c519f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:35.956834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3cccc-70bf-492c-9b5d-75f5757ae3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:35.958227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3cccc-70bf-492c-9b5d-75f5757ae3ff None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:35.958742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3cccc-70bf-492c-9b5d-75f5757ae3ff None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:35.959244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3cccc-70bf-492c-9b5d-75f5757ae3ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:35.978558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3cccc-70bf-492c-9b5d-75f5757ae3ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-35d3cccc-70bf-492c-9b5d-75f5757ae3ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:35.994761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:35.995997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:35.997505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:35.999102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 292/580] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:36.000236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3cccc-70bf-492c-9b5d-75f5757ae3ff None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:36.018682 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5384cd7-e7dd-4139-a3b2-241c32e2557f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:36.066138 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406d4db-2b3a-4abe-b6b2-aba6973733f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:36.066849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406d4db-2b3a-4abe-b6b2-aba6973733f0 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:36.067177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406d4db-2b3a-4abe-b6b2-aba6973733f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:36.067521 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406d4db-2b3a-4abe-b6b2-aba6973733f0 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:36.068366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406d4db-2b3a-4abe-b6b2-aba6973733f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:36.068726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406d4db-2b3a-4abe-b6b2-aba6973733f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:36.070611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406d4db-2b3a-4abe-b6b2-aba6973733f0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:36.092165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 289/581] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:36.092165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406d4db-2b3a-4abe-b6b2-aba6973733f0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:36.132168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3cccc-70bf-492c-9b5d-75f5757ae3ff None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:36.172513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223db61c-038f-4cbf-9bac-fb06eb5a249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:36.173360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223db61c-038f-4cbf-9bac-fb06eb5a249f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:36.173818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223db61c-038f-4cbf-9bac-fb06eb5a249f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:36.174257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223db61c-038f-4cbf-9bac-fb06eb5a249f None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:36.175301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223db61c-038f-4cbf-9bac-fb06eb5a249f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:36.175785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223db61c-038f-4cbf-9bac-fb06eb5a249f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:36.178110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223db61c-038f-4cbf-9bac-fb06eb5a249f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:36.197870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 293/582] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:36.197870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223db61c-038f-4cbf-9bac-fb06eb5a249f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:36.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406d4db-2b3a-4abe-b6b2-aba6973733f0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:36.305690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ee67b-6005-45ad-a7d1-cdb2c1ecf382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:36.307052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ee67b-6005-45ad-a7d1-cdb2c1ecf382 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:36.307502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ee67b-6005-45ad-a7d1-cdb2c1ecf382 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:36.307982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ee67b-6005-45ad-a7d1-cdb2c1ecf382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:36.327465 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ee67b-6005-45ad-a7d1-cdb2c1ecf382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:36.330096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ee67b-6005-45ad-a7d1-cdb2c1ecf382 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-2c7ee67b-6005-45ad-a7d1-cdb2c1ecf382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:36.359574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:36.361075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:36.363424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:36.363424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:36.363424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:36.363424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:36.363424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 290/583] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:36.363424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ee67b-6005-45ad-a7d1-cdb2c1ecf382 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:36.372303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223db61c-038f-4cbf-9bac-fb06eb5a249f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:36.425103 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0c83c-10ea-4168-90a9-a1468589a39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:36.426375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0c83c-10ea-4168-90a9-a1468589a39c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:36.426952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0c83c-10ea-4168-90a9-a1468589a39c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:36.427503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0c83c-10ea-4168-90a9-a1468589a39c None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:36.429235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0c83c-10ea-4168-90a9-a1468589a39c None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:36.429773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0c83c-10ea-4168-90a9-a1468589a39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:36.432421 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0c83c-10ea-4168-90a9-a1468589a39c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:36.454768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 294/584] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:36.454768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0c83c-10ea-4168-90a9-a1468589a39c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:36.538423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ee67b-6005-45ad-a7d1-cdb2c1ecf382 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:36.601439 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac8aee-1fe8-42a9-a154-8ecc02cda8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:36.603080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac8aee-1fe8-42a9-a154-8ecc02cda8d9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:36.603421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac8aee-1fe8-42a9-a154-8ecc02cda8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:36.603914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac8aee-1fe8-42a9-a154-8ecc02cda8d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:36.610079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0c83c-10ea-4168-90a9-a1468589a39c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:36.613975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac8aee-1fe8-42a9-a154-8ecc02cda8d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:36.616468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac8aee-1fe8-42a9-a154-8ecc02cda8d9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-dbac8aee-1fe8-42a9-a154-8ecc02cda8d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:36.635543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:36.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:36.639491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:36.639491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:36.639491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:36.639491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:36.639491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:36.639491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:36.639491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:36.639491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:36.639491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 291/585] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:36.639491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac8aee-1fe8-42a9-a154-8ecc02cda8d9 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:36.655306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe646e15-9e9a-4032-bc90-e147cab2c251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:36.656585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe646e15-9e9a-4032-bc90-e147cab2c251 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:36.656997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe646e15-9e9a-4032-bc90-e147cab2c251 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:36.657403 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe646e15-9e9a-4032-bc90-e147cab2c251 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:36.666267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe646e15-9e9a-4032-bc90-e147cab2c251 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:36.668005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe646e15-9e9a-4032-bc90-e147cab2c251 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-fe646e15-9e9a-4032-bc90-e147cab2c251 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:36.694645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:36.696734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:36.698784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:36.698784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:36.698784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:36.698784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:36.698784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:36.698784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:36.698784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:36.698784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:36.709243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 295/586] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:36.709243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe646e15-9e9a-4032-bc90-e147cab2c251 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:36.810441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac8aee-1fe8-42a9-a154-8ecc02cda8d9 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:36.846253 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe646e15-9e9a-4032-bc90-e147cab2c251 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:36.867556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719c301-faba-44c1-af8f-8ceb592d1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:36.868578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719c301-faba-44c1-af8f-8ceb592d1f01 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:36.869086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719c301-faba-44c1-af8f-8ceb592d1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:36.869565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719c301-faba-44c1-af8f-8ceb592d1f01 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:36.870702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719c301-faba-44c1-af8f-8ceb592d1f01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:36.871259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719c301-faba-44c1-af8f-8ceb592d1f01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:36.873424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719c301-faba-44c1-af8f-8ceb592d1f01 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:36.894596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 292/587] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:36.896991 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719c301-faba-44c1-af8f-8ceb592d1f01 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:36.900597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b0016-0ad0-494a-a1ca-e6ca8a642c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:36.902381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b0016-0ad0-494a-a1ca-e6ca8a642c7e None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:36.902809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b0016-0ad0-494a-a1ca-e6ca8a642c7e None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:36.903296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b0016-0ad0-494a-a1ca-e6ca8a642c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:36.929116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b0016-0ad0-494a-a1ca-e6ca8a642c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:36.932056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b0016-0ad0-494a-a1ca-e6ca8a642c7e None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-262b0016-0ad0-494a-a1ca-e6ca8a642c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:36.964121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:36.965539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:36.966433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:36.966433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:36.966433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:36.966433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:36.975230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 296/588] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:36.976567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b0016-0ad0-494a-a1ca-e6ca8a642c7e None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:37.088961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719c301-faba-44c1-af8f-8ceb592d1f01 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:37.107271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b0016-0ad0-494a-a1ca-e6ca8a642c7e None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:37.135527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983278-8a8d-4d2f-b707-ccb637656394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:37.136309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983278-8a8d-4d2f-b707-ccb637656394 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:37.136592 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983278-8a8d-4d2f-b707-ccb637656394 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:37.136865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983278-8a8d-4d2f-b707-ccb637656394 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:37.137670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13983278-8a8d-4d2f-b707-ccb637656394 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:37.137981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13983278-8a8d-4d2f-b707-ccb637656394 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:37.139494 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13983278-8a8d-4d2f-b707-ccb637656394 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:37.153908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789686a-0042-4524-bc6a-0293b16ae0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:37.154126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 293/589] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:37.155011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789686a-0042-4524-bc6a-0293b16ae0b1 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:37.155159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983278-8a8d-4d2f-b707-ccb637656394 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:37.155628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789686a-0042-4524-bc6a-0293b16ae0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:37.155903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789686a-0042-4524-bc6a-0293b16ae0b1 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:37.156833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789686a-0042-4524-bc6a-0293b16ae0b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:37.157136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789686a-0042-4524-bc6a-0293b16ae0b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:37.158602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789686a-0042-4524-bc6a-0293b16ae0b1 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:37.179111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 297/590] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:37.180770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789686a-0042-4524-bc6a-0293b16ae0b1 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:37.334610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983278-8a8d-4d2f-b707-ccb637656394 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:37.362587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789686a-0042-4524-bc6a-0293b16ae0b1 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:37.398557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaa90c-deb5-42e2-939d-84879ff2104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:37.399545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaa90c-deb5-42e2-939d-84879ff2104c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:37.399857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaa90c-deb5-42e2-939d-84879ff2104c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:37.400042 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaa90c-deb5-42e2-939d-84879ff2104c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:37.409069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5387d-238b-4915-b1ad-a32d56360cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:37.409069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5387d-238b-4915-b1ad-a32d56360cb8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:37.409497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5387d-238b-4915-b1ad-a32d56360cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:37.409629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5387d-238b-4915-b1ad-a32d56360cb8 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:37.410367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5387d-238b-4915-b1ad-a32d56360cb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:37.410675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5387d-238b-4915-b1ad-a32d56360cb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:37.411048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecaa90c-deb5-42e2-939d-84879ff2104c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:37.412684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5387d-238b-4915-b1ad-a32d56360cb8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:37.413510 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecaa90c-deb5-42e2-939d-84879ff2104c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:37.431036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 298/591] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:37.431036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5387d-238b-4915-b1ad-a32d56360cb8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-5ecaa90c-deb5-42e2-939d-84879ff2104c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:37.431608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:37.433223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 294/592] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:37.434668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaa90c-deb5-42e2-939d-84879ff2104c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:37.584568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5387d-238b-4915-b1ad-a32d56360cb8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:37.593667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaa90c-deb5-42e2-939d-84879ff2104c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:37.633050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315c91c-ec28-454b-b0fd-d7da2a912394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:37.633963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315c91c-ec28-454b-b0fd-d7da2a912394 None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:37.634256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315c91c-ec28-454b-b0fd-d7da2a912394 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:37.634503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315c91c-ec28-454b-b0fd-d7da2a912394 None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/cc4f45e859d84ceaa5a01477e83e098e/roles/879ba54490a944d6abcf2ac5e2f1bea9` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:37.653887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315c91c-ec28-454b-b0fd-d7da2a912394 None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=cc4f45e859d84ceaa5a01477e83e098e, role_id=879ba54490a944d6abcf2ac5e2f1bea9)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:37.655375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315c91c-ec28-454b-b0fd-d7da2a912394 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:37.655763 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f3512-38be-4de3-92dd-e8d1cb4cded9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:37.657282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f3512-38be-4de3-92dd-e8d1cb4cded9 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:37.657746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f3512-38be-4de3-92dd-e8d1cb4cded9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:37.658198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f3512-38be-4de3-92dd-e8d1cb4cded9 None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/24ad0dd9d265425087be2be8d245b7fc/roles/879ba54490a944d6abcf2ac5e2f1bea9` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:37.679251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 299/593] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:41:37 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/cc4f45e859d84ceaa5a01477e83e098e/roles/879ba54490a944d6abcf2ac5e2f1bea9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:41:37.680504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315c91c-ec28-454b-b0fd-d7da2a912394 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:37.694238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f3512-38be-4de3-92dd-e8d1cb4cded9 None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=24ad0dd9d265425087be2be8d245b7fc, role_id=879ba54490a944d6abcf2ac5e2f1bea9)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:37.698048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f3512-38be-4de3-92dd-e8d1cb4cded9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:37.734964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 295/594] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:41:37 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/24ad0dd9d265425087be2be8d245b7fc/roles/879ba54490a944d6abcf2ac5e2f1bea9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:41:37.734964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f3512-38be-4de3-92dd-e8d1cb4cded9 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:37.882592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315c91c-ec28-454b-b0fd-d7da2a912394 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:37.896447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f3512-38be-4de3-92dd-e8d1cb4cded9 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:37.931815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb07e2-230b-46ec-b01d-68069f6ed2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:37.932649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb07e2-230b-46ec-b01d-68069f6ed2d9 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:37.932932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb07e2-230b-46ec-b01d-68069f6ed2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:37.933193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb07e2-230b-46ec-b01d-68069f6ed2d9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:37.942200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb07e2-230b-46ec-b01d-68069f6ed2d9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:37.943928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb07e2-230b-46ec-b01d-68069f6ed2d9 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:37.945972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb10c4-a9b1-471e-be54-e9798be415fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:37.947149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb10c4-a9b1-471e-be54-e9798be415fc None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:37.947664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb10c4-a9b1-471e-be54-e9798be415fc None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:37.947938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb10c4-a9b1-471e-be54-e9798be415fc None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:37.948888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb10c4-a9b1-471e-be54-e9798be415fc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:37.949198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb10c4-a9b1-471e-be54-e9798be415fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:37.951365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb10c4-a9b1-471e-be54-e9798be415fc None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-befb07e2-230b-46ec-b01d-68069f6ed2d9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:37.962896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:37.965072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 300/595] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4c96-99b2-4db6-94a4-7648cc7e431c None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:37.966264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4c96-99b2-4db6-94a4-7648cc7e431c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:37.967006 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4c96-99b2-4db6-94a4-7648cc7e431c None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:37.970745 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 301/596] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:37.975765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07287fc7-c350-42dd-865b-65c28783ad19 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:37.975765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07287fc7-c350-42dd-865b-65c28783ad19 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:37.975765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07287fc7-c350-42dd-865b-65c28783ad19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:37.981816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 296/597] 10.222.0.79 () {56 vars in 1137 bytes} [Fri May 13 13:41:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:37.983524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb10c4-a9b1-471e-be54-e9798be415fc None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:38.008623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-07287fc7-c350-42dd-865b-65c28783ad19 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:38.009876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-07287fc7-c350-42dd-865b-65c28783ad19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:38.094926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 302/598] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:38.095340 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07287fc7-c350-42dd-865b-65c28783ad19 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:38.147103 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb10c4-a9b1-471e-be54-e9798be415fc None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:38.190767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464d467-e918-4f7e-b54a-1ad91ff87ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:38.191528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464d467-e918-4f7e-b54a-1ad91ff87ab5 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:38.191803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464d467-e918-4f7e-b54a-1ad91ff87ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:38.192182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464d467-e918-4f7e-b54a-1ad91ff87ab5 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:38.193824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464d467-e918-4f7e-b54a-1ad91ff87ab5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:38.194228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464d467-e918-4f7e-b54a-1ad91ff87ab5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:38.195739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464d467-e918-4f7e-b54a-1ad91ff87ab5 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:38.224028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07287fc7-c350-42dd-865b-65c28783ad19 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:38.224306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 297/599] 10.222.0.79 () {56 vars in 1139 bytes} [Fri May 13 13:41:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:38.225206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464d467-e918-4f7e-b54a-1ad91ff87ab5 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:38.269814 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75267f73-5eee-4fd7-95b9-59b5b7735161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:38.270798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75267f73-5eee-4fd7-95b9-59b5b7735161 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:38.271090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75267f73-5eee-4fd7-95b9-59b5b7735161 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:38.271358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75267f73-5eee-4fd7-95b9-59b5b7735161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:38.284477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75267f73-5eee-4fd7-95b9-59b5b7735161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:38.287807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75267f73-5eee-4fd7-95b9-59b5b7735161 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-75267f73-5eee-4fd7-95b9-59b5b7735161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:38.316226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:38.317257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:38.319144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:38.319144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:38.319144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:38.319144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:38.319144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 303/600] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:38.320237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75267f73-5eee-4fd7-95b9-59b5b7735161 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:38.346945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464d467-e918-4f7e-b54a-1ad91ff87ab5 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:38.384895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec19f1-e78c-4f49-86be-d9a36dadb251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:38.385693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec19f1-e78c-4f49-86be-d9a36dadb251 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:38.385969 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec19f1-e78c-4f49-86be-d9a36dadb251 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:38.386240 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec19f1-e78c-4f49-86be-d9a36dadb251 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:38.397291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec19f1-e78c-4f49-86be-d9a36dadb251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-18ec19f1-e78c-4f49-86be-d9a36dadb251 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:38.405294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:38.408654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:38.412363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 298/601] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:38.424580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec19f1-e78c-4f49-86be-d9a36dadb251 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:38.426286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75267f73-5eee-4fd7-95b9-59b5b7735161 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:38.462592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7a42d-1885-4ebe-b460-0e0442d9b918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:38.463534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7a42d-1885-4ebe-b460-0e0442d9b918 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:38.463816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7a42d-1885-4ebe-b460-0e0442d9b918 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:38.464200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7a42d-1885-4ebe-b460-0e0442d9b918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:38.476539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7a42d-1885-4ebe-b460-0e0442d9b918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:38.478099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7a42d-1885-4ebe-b460-0e0442d9b918 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-6ad7a42d-1885-4ebe-b460-0e0442d9b918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:38.497902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:38.500125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:38.502210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:38.502210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:38.502210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:38.502210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:38.502210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 304/602] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:38.502210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7a42d-1885-4ebe-b460-0e0442d9b918 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:38.527868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec19f1-e78c-4f49-86be-d9a36dadb251 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:38.566570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9995e11-74e5-4f32-8392-d5527a258a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:38.567419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9995e11-74e5-4f32-8392-d5527a258a63 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:38.567751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9995e11-74e5-4f32-8392-d5527a258a63 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:38.568029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9995e11-74e5-4f32-8392-d5527a258a63 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:38.568756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9995e11-74e5-4f32-8392-d5527a258a63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:38.569117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9995e11-74e5-4f32-8392-d5527a258a63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:38.570523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9995e11-74e5-4f32-8392-d5527a258a63 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:38.584492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 299/603] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:38.585917 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9995e11-74e5-4f32-8392-d5527a258a63 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:38.608546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7a42d-1885-4ebe-b460-0e0442d9b918 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:38.645106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f774ef-38ab-482f-a9db-ed7e12b48f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:38.646760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f774ef-38ab-482f-a9db-ed7e12b48f82 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:38.647161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f774ef-38ab-482f-a9db-ed7e12b48f82 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:38.647549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f774ef-38ab-482f-a9db-ed7e12b48f82 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:38.648865 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f774ef-38ab-482f-a9db-ed7e12b48f82 None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:38.649395 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f774ef-38ab-482f-a9db-ed7e12b48f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:38.652255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f774ef-38ab-482f-a9db-ed7e12b48f82 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:38.666337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 305/604] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:38.667261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f774ef-38ab-482f-a9db-ed7e12b48f82 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:38.719039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9995e11-74e5-4f32-8392-d5527a258a63 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:38.755171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dc79e-4aac-4029-abba-a040b2451e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:38.755842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dc79e-4aac-4029-abba-a040b2451e60 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:38.756124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dc79e-4aac-4029-abba-a040b2451e60 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:38.756491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dc79e-4aac-4029-abba-a040b2451e60 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:38.757488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dc79e-4aac-4029-abba-a040b2451e60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:38.758034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dc79e-4aac-4029-abba-a040b2451e60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:38.760321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dc79e-4aac-4029-abba-a040b2451e60 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:38.782752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 300/605] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:38.784368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dc79e-4aac-4029-abba-a040b2451e60 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:38.820111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f774ef-38ab-482f-a9db-ed7e12b48f82 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:38.858246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0371c-5fda-439f-bb9d-6208488f3f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:38.859139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0371c-5fda-439f-bb9d-6208488f3f47 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:38.859423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0371c-5fda-439f-bb9d-6208488f3f47 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:38.859700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0371c-5fda-439f-bb9d-6208488f3f47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:38.867396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0371c-5fda-439f-bb9d-6208488f3f47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:38.868879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0371c-5fda-439f-bb9d-6208488f3f47 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-e1d0371c-5fda-439f-bb9d-6208488f3f47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:38.885516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:38.887685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:38.890441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:38.890441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:38.890441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:38.890441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:38.890441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:38.890441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:38.890441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:38.890441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:38.890441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 306/606] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:38.890441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0371c-5fda-439f-bb9d-6208488f3f47 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:38.915906 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dc79e-4aac-4029-abba-a040b2451e60 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:38.951404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00142e9f-dfde-4df4-bdac-94c7d3b3321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:38.952208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00142e9f-dfde-4df4-bdac-94c7d3b3321e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:38.952488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00142e9f-dfde-4df4-bdac-94c7d3b3321e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:38.952762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00142e9f-dfde-4df4-bdac-94c7d3b3321e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:38.964485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00142e9f-dfde-4df4-bdac-94c7d3b3321e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:38.965948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00142e9f-dfde-4df4-bdac-94c7d3b3321e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-00142e9f-dfde-4df4-bdac-94c7d3b3321e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:38.987619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:38.988599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:38.989543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:38.989543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:38.989543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:38.989543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:38.993791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 301/607] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:38.993967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0371c-5fda-439f-bb9d-6208488f3f47 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:38.994777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00142e9f-dfde-4df4-bdac-94c7d3b3321e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:39.030011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42309f0b-a1ea-41e4-9678-2fe96693d46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:39.030869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42309f0b-a1ea-41e4-9678-2fe96693d46f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:39.031149 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42309f0b-a1ea-41e4-9678-2fe96693d46f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:39.031426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42309f0b-a1ea-41e4-9678-2fe96693d46f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:39.041027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42309f0b-a1ea-41e4-9678-2fe96693d46f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:39.043132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42309f0b-a1ea-41e4-9678-2fe96693d46f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-42309f0b-a1ea-41e4-9678-2fe96693d46f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:39.069717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:39.071902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:39.074250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:39.074250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:39.074250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:39.074250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:39.074250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:39.074250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:39.074250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:39.074250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:39.074250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 307/608] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:39.074250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42309f0b-a1ea-41e4-9678-2fe96693d46f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:39.102445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00142e9f-dfde-4df4-bdac-94c7d3b3321e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:39.137661 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4e8b2-db43-4fe3-8c34-2e9bde955a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:39.138333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e8b2-db43-4fe3-8c34-2e9bde955a59 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:39.138614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e8b2-db43-4fe3-8c34-2e9bde955a59 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:39.138928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e8b2-db43-4fe3-8c34-2e9bde955a59 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:39.139719 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4e8b2-db43-4fe3-8c34-2e9bde955a59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:39.140029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4e8b2-db43-4fe3-8c34-2e9bde955a59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:39.141489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4e8b2-db43-4fe3-8c34-2e9bde955a59 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:39.155442 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 302/609] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:39.156320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4e8b2-db43-4fe3-8c34-2e9bde955a59 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:39.212402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42309f0b-a1ea-41e4-9678-2fe96693d46f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:39.249017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2d4fb-a831-452b-97b8-f7ee69d1d774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:39.249695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2d4fb-a831-452b-97b8-f7ee69d1d774 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:39.249983 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2d4fb-a831-452b-97b8-f7ee69d1d774 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:39.250263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2d4fb-a831-452b-97b8-f7ee69d1d774 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:39.251051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2d4fb-a831-452b-97b8-f7ee69d1d774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:39.251545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2d4fb-a831-452b-97b8-f7ee69d1d774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:39.252972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2d4fb-a831-452b-97b8-f7ee69d1d774 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:39.269727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 308/610] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:39.270656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2d4fb-a831-452b-97b8-f7ee69d1d774 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:39.316507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4e8b2-db43-4fe3-8c34-2e9bde955a59 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:39.376596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80994e4-7a59-4758-b56d-ed9f7a683ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:39.377823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80994e4-7a59-4758-b56d-ed9f7a683ca8 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:39.379230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80994e4-7a59-4758-b56d-ed9f7a683ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:39.379761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80994e4-7a59-4758-b56d-ed9f7a683ca8 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:39.381941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80994e4-7a59-4758-b56d-ed9f7a683ca8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:39.382501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80994e4-7a59-4758-b56d-ed9f7a683ca8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:39.385289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80994e4-7a59-4758-b56d-ed9f7a683ca8 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:39.405758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 303/611] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:39.406834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80994e4-7a59-4758-b56d-ed9f7a683ca8 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:39.451307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2d4fb-a831-452b-97b8-f7ee69d1d774 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:39.507600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24e62d-649d-4f4d-9847-1d7b2f20d169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:39.508971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24e62d-649d-4f4d-9847-1d7b2f20d169 None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:39.509466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24e62d-649d-4f4d-9847-1d7b2f20d169 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:39.509959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24e62d-649d-4f4d-9847-1d7b2f20d169 None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/e03fa33a68ff4567bc25c9611cd9ae8b/roles/879ba54490a944d6abcf2ac5e2f1bea9` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:39.528670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24e62d-649d-4f4d-9847-1d7b2f20d169 None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=e03fa33a68ff4567bc25c9611cd9ae8b, role_id=879ba54490a944d6abcf2ac5e2f1bea9)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:39.530488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24e62d-649d-4f4d-9847-1d7b2f20d169 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:39.553188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 309/612] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:41:39 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/e03fa33a68ff4567bc25c9611cd9ae8b/roles/879ba54490a944d6abcf2ac5e2f1bea9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:41:39.554987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24e62d-649d-4f4d-9847-1d7b2f20d169 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:39.559603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80994e4-7a59-4758-b56d-ed9f7a683ca8 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:39.607011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3d344-5fc8-4650-87cf-396cb69c5776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:39.607916 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3d344-5fc8-4650-87cf-396cb69c5776 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:39.608368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3d344-5fc8-4650-87cf-396cb69c5776 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:39.608905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3d344-5fc8-4650-87cf-396cb69c5776 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:39.619709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3d344-5fc8-4650-87cf-396cb69c5776 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:39.621771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3d344-5fc8-4650-87cf-396cb69c5776 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-34c3d344-5fc8-4650-87cf-396cb69c5776 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:39.639445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:39.641277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:39.643163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 304/613] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:41:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:39.647086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3d344-5fc8-4650-87cf-396cb69c5776 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:39.725264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24e62d-649d-4f4d-9847-1d7b2f20d169 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:39.770617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d446a-f5f0-4ed6-b364-cff51be384af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:39.772472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d446a-f5f0-4ed6-b364-cff51be384af None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:39.773130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d446a-f5f0-4ed6-b364-cff51be384af None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:39.773637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d446a-f5f0-4ed6-b364-cff51be384af None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/8f784cc461ab48ad92140680758a3704/roles/879ba54490a944d6abcf2ac5e2f1bea9` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:39.810166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d446a-f5f0-4ed6-b364-cff51be384af None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=8f784cc461ab48ad92140680758a3704, role_id=879ba54490a944d6abcf2ac5e2f1bea9)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:39.812501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d446a-f5f0-4ed6-b364-cff51be384af None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:39.817930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3d344-5fc8-4650-87cf-396cb69c5776 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:39.842154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 310/614] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:41:39 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/8f784cc461ab48ad92140680758a3704/roles/879ba54490a944d6abcf2ac5e2f1bea9 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:41:39.872642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288cfc3-2df8-494e-943e-4973b2d984e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:39.874320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288cfc3-2df8-494e-943e-4973b2d984e6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:39.874986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288cfc3-2df8-494e-943e-4973b2d984e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:39.875550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288cfc3-2df8-494e-943e-4973b2d984e6 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:39.876833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288cfc3-2df8-494e-943e-4973b2d984e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:39.877409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288cfc3-2df8-494e-943e-4973b2d984e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:39.880410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288cfc3-2df8-494e-943e-4973b2d984e6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:39.901933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 305/615] 10.222.0.79 () {56 vars in 1135 bytes} [Fri May 13 13:41:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:39.907268 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d446a-f5f0-4ed6-b364-cff51be384af None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:40.059270 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d446a-f5f0-4ed6-b364-cff51be384af None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:40.108632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170dac3-c98a-4619-996a-858faed1995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:40.109514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170dac3-c98a-4619-996a-858faed1995e None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:40.109803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170dac3-c98a-4619-996a-858faed1995e None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:40.110077 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170dac3-c98a-4619-996a-858faed1995e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:40.128839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170dac3-c98a-4619-996a-858faed1995e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:40.130353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170dac3-c98a-4619-996a-858faed1995e None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-5170dac3-c98a-4619-996a-858faed1995e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:40.154423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:40.156226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:40.157992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:40.157992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:40.157992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:40.157992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:40.157992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 311/616] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:40.164606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288cfc3-2df8-494e-943e-4973b2d984e6 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:40.295428 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288cfc3-2df8-494e-943e-4973b2d984e6 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:40.347216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e1602-8bd8-4b7c-98f5-64c86f7425af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:40.347932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e1602-8bd8-4b7c-98f5-64c86f7425af None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:40.348270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e1602-8bd8-4b7c-98f5-64c86f7425af None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:40.349147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e1602-8bd8-4b7c-98f5-64c86f7425af None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:40.349896 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e1602-8bd8-4b7c-98f5-64c86f7425af None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:40.350205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e1602-8bd8-4b7c-98f5-64c86f7425af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:40.351786 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e1602-8bd8-4b7c-98f5-64c86f7425af None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:40.370967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 306/617] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:40.377568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170dac3-c98a-4619-996a-858faed1995e None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:40.586187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170dac3-c98a-4619-996a-858faed1995e None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:40.646441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9d510-44fe-4212-8c80-306dcf915394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:40.647695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9d510-44fe-4212-8c80-306dcf915394 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:40.648261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9d510-44fe-4212-8c80-306dcf915394 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:40.649008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9d510-44fe-4212-8c80-306dcf915394 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:40.668781 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9d510-44fe-4212-8c80-306dcf915394 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:40.671301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9d510-44fe-4212-8c80-306dcf915394 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-8aa9d510-44fe-4212-8c80-306dcf915394 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:40.702060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:40.703085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:40.703978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:40.703978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:40.703978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:40.703978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:40.703978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:40.703978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:40.703978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:40.703978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:40.708818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 312/618] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:40.716987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e1602-8bd8-4b7c-98f5-64c86f7425af None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:40.971684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e1602-8bd8-4b7c-98f5-64c86f7425af None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:41.018123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdab69-879a-4fec-85f7-fd043e35be87 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:41.019046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdab69-879a-4fec-85f7-fd043e35be87 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:41.019864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdab69-879a-4fec-85f7-fd043e35be87 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:41.021639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415687a6-8325-4a50-ae31-ae89dd0732bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:41.022799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415687a6-8325-4a50-ae31-ae89dd0732bb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:41.026825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415687a6-8325-4a50-ae31-ae89dd0732bb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:41.026825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415687a6-8325-4a50-ae31-ae89dd0732bb None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:41.026825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415687a6-8325-4a50-ae31-ae89dd0732bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:41.026825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415687a6-8325-4a50-ae31-ae89dd0732bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:41.027750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 313/619] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:41.027918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415687a6-8325-4a50-ae31-ae89dd0732bb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:41.030603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72778c-6422-4cee-a2fa-3113725a1965 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:41.031507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72778c-6422-4cee-a2fa-3113725a1965 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:41.032102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72778c-6422-4cee-a2fa-3113725a1965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:41.051024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 307/620] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:41.067583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415687a6-8325-4a50-ae31-ae89dd0732bb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:41.070051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-ab72778c-6422-4cee-a2fa-3113725a1965 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:41.071538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ab72778c-6422-4cee-a2fa-3113725a1965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:41.269793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 314/621] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:41.272573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d496a-c02a-423d-a41c-d8a07db1fa09 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:41.273255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d496a-c02a-423d-a41c-d8a07db1fa09 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:41.273580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d496a-c02a-423d-a41c-d8a07db1fa09 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:41.288872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 315/622] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:41.292057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172d261-2eb6-49bb-815d-ec8f2336fe18 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:41.292647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172d261-2eb6-49bb-815d-ec8f2336fe18 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:41.293054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172d261-2eb6-49bb-815d-ec8f2336fe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:41.326283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415687a6-8325-4a50-ae31-ae89dd0732bb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:41.342837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-3172d261-2eb6-49bb-815d-ec8f2336fe18 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:41.344098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-3172d261-2eb6-49bb-815d-ec8f2336fe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:41.378191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66741e97-3c80-4866-802b-c2df38915845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:41.379211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66741e97-3c80-4866-802b-c2df38915845 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:41.379488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66741e97-3c80-4866-802b-c2df38915845 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:41.380364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66741e97-3c80-4866-802b-c2df38915845 None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/4139cfe906bd4c7d94226ab589577627/roles/879ba54490a944d6abcf2ac5e2f1bea9` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:41.400724 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66741e97-3c80-4866-802b-c2df38915845 None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=4139cfe906bd4c7d94226ab589577627, role_id=879ba54490a944d6abcf2ac5e2f1bea9)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:41.402524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66741e97-3c80-4866-802b-c2df38915845 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:41.434744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 308/623] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:41:41 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/4139cfe906bd4c7d94226ab589577627/roles/879ba54490a944d6abcf2ac5e2f1bea9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:41:41.434744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66741e97-3c80-4866-802b-c2df38915845 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:41.454480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 316/624] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:41:41.521420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172d261-2eb6-49bb-815d-ec8f2336fe18 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:41.573706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66741e97-3c80-4866-802b-c2df38915845 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:41.628713 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1999f5b-ede2-4ed1-92aa-b8a462ae0360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:41.630247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1999f5b-ede2-4ed1-92aa-b8a462ae0360 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:41.630604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1999f5b-ede2-4ed1-92aa-b8a462ae0360 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:41.630984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1999f5b-ede2-4ed1-92aa-b8a462ae0360 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:41.646189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1999f5b-ede2-4ed1-92aa-b8a462ae0360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-c1999f5b-ede2-4ed1-92aa-b8a462ae0360 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:41.656509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:41.657599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:41.658756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 309/625] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:41.670299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1999f5b-ede2-4ed1-92aa-b8a462ae0360 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:41.740660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172d261-2eb6-49bb-815d-ec8f2336fe18 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:41.786571 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca47393-e2c9-492d-bb04-f1998368ae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:41.787778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47393-e2c9-492d-bb04-f1998368ae42 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:41.788275 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47393-e2c9-492d-bb04-f1998368ae42 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:41.788722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47393-e2c9-492d-bb04-f1998368ae42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:41.803296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca47393-e2c9-492d-bb04-f1998368ae42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-bca47393-e2c9-492d-bb04-f1998368ae42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:41.811552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:41.812543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:41.813374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:41.816386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 317/626] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:41.822397 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca47393-e2c9-492d-bb04-f1998368ae42 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:41.830744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1999f5b-ede2-4ed1-92aa-b8a462ae0360 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:41.871582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fda02-0206-49c9-8fdb-447e6f15b691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:41.872659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fda02-0206-49c9-8fdb-447e6f15b691 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:41.873156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fda02-0206-49c9-8fdb-447e6f15b691 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:41.873649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fda02-0206-49c9-8fdb-447e6f15b691 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:41.874593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fda02-0206-49c9-8fdb-447e6f15b691 None admin] RBAC: Adding query filter params (name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:41.875115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fda02-0206-49c9-8fdb-447e6f15b691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:41.877584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fda02-0206-49c9-8fdb-447e6f15b691 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:41.893748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 310/627] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:41.895666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fda02-0206-49c9-8fdb-447e6f15b691 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:42.029464 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fda02-0206-49c9-8fdb-447e6f15b691 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:42.070481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d40b10-00f1-46c6-a066-f36c96289105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:42.071615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d40b10-00f1-46c6-a066-f36c96289105 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:42.072082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d40b10-00f1-46c6-a066-f36c96289105 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:42.072540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d40b10-00f1-46c6-a066-f36c96289105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:42.100073 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d40b10-00f1-46c6-a066-f36c96289105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:42.101443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d40b10-00f1-46c6-a066-f36c96289105 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:42.104681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca47393-e2c9-492d-bb04-f1998368ae42 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-a4d40b10-00f1-46c6-a066-f36c96289105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:42.139469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:42.140709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:42.141834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:42.141834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:42.141834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:42.141834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:42.151244 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 311/628] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:42.160067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a762c-4c16-451b-9b3f-aa5fd3e4ca30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:42.161108 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a762c-4c16-451b-9b3f-aa5fd3e4ca30 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:42.161711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a762c-4c16-451b-9b3f-aa5fd3e4ca30 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:42.162178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a762c-4c16-451b-9b3f-aa5fd3e4ca30 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:42.165208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a762c-4c16-451b-9b3f-aa5fd3e4ca30 None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:42.166344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a762c-4c16-451b-9b3f-aa5fd3e4ca30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:42.168289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a762c-4c16-451b-9b3f-aa5fd3e4ca30 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:42.171971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d40b10-00f1-46c6-a066-f36c96289105 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:42.182982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 318/629] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:42.201895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a762c-4c16-451b-9b3f-aa5fd3e4ca30 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:42.360101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d40b10-00f1-46c6-a066-f36c96289105 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:42.435224 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa33b02-380b-40f5-bb17-fcc58a8e4c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:42.436394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa33b02-380b-40f5-bb17-fcc58a8e4c8e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:42.436897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa33b02-380b-40f5-bb17-fcc58a8e4c8e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:42.437403 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa33b02-380b-40f5-bb17-fcc58a8e4c8e None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:42.438842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa33b02-380b-40f5-bb17-fcc58a8e4c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:42.439405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa33b02-380b-40f5-bb17-fcc58a8e4c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:42.442067 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa33b02-380b-40f5-bb17-fcc58a8e4c8e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:42.472777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 312/630] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:42.482273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa33b02-380b-40f5-bb17-fcc58a8e4c8e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:42.505546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a762c-4c16-451b-9b3f-aa5fd3e4ca30 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:42.601967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a638186-126e-4aaf-8903-69460a6e5916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:42.603328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a638186-126e-4aaf-8903-69460a6e5916 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:42.603816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a638186-126e-4aaf-8903-69460a6e5916 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:42.604320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a638186-126e-4aaf-8903-69460a6e5916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:42.633344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a638186-126e-4aaf-8903-69460a6e5916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:42.634760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a638186-126e-4aaf-8903-69460a6e5916 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-1a638186-126e-4aaf-8903-69460a6e5916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:42.678233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:42.679815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:42.682392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:42.682392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:42.682392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:42.682392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:42.682392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 319/631] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 480 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:42.688985 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a638186-126e-4aaf-8903-69460a6e5916 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:42.735271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa33b02-380b-40f5-bb17-fcc58a8e4c8e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:42.805177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee092f-e872-4e34-b9d0-1ade92a88632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:42.806002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee092f-e872-4e34-b9d0-1ade92a88632 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:42.806270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee092f-e872-4e34-b9d0-1ade92a88632 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:42.806542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee092f-e872-4e34-b9d0-1ade92a88632 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:42.820448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee092f-e872-4e34-b9d0-1ade92a88632 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:42.821955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee092f-e872-4e34-b9d0-1ade92a88632 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-22ee092f-e872-4e34-b9d0-1ade92a88632 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:42.850949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:42.853234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:42.857090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 313/632] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:42.859272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee092f-e872-4e34-b9d0-1ade92a88632 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:42.947203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a638186-126e-4aaf-8903-69460a6e5916 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:43.022542 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee6bf2-c3a4-4668-83a0-ee8bcad2db0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:43.023261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee6bf2-c3a4-4668-83a0-ee8bcad2db0d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:43.023538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee6bf2-c3a4-4668-83a0-ee8bcad2db0d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:43.023815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee6bf2-c3a4-4668-83a0-ee8bcad2db0d None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:43.024560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee6bf2-c3a4-4668-83a0-ee8bcad2db0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:43.024859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee6bf2-c3a4-4668-83a0-ee8bcad2db0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:43.026328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee6bf2-c3a4-4668-83a0-ee8bcad2db0d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:43.050237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 320/633] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:43.057250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee6bf2-c3a4-4668-83a0-ee8bcad2db0d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:43.126971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee092f-e872-4e34-b9d0-1ade92a88632 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:43.176349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91ef31-4cdd-4f9b-8855-c77fa83242ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:43.177213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91ef31-4cdd-4f9b-8855-c77fa83242ec None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:43.177494 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91ef31-4cdd-4f9b-8855-c77fa83242ec None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:43.177768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91ef31-4cdd-4f9b-8855-c77fa83242ec None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:43.178628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91ef31-4cdd-4f9b-8855-c77fa83242ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:43.178972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91ef31-4cdd-4f9b-8855-c77fa83242ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:43.184117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91ef31-4cdd-4f9b-8855-c77fa83242ec None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:43.211530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee6bf2-c3a4-4668-83a0-ee8bcad2db0d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:43.214811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 314/634] 10.222.0.79 () {56 vars in 1137 bytes} [Fri May 13 13:41:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:43.216694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9598e-bb37-4c9f-83b6-3fbabeb9f8b1 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:43.216975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9598e-bb37-4c9f-83b6-3fbabeb9f8b1 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:43.217258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9598e-bb37-4c9f-83b6-3fbabeb9f8b1 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:43.218636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 315/635] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:43.219436 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9598e-bb37-4c9f-83b6-3fbabeb9f8b1 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:43.274930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2762a-d17e-4374-ba1a-08cfe45e321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:43.276146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2762a-d17e-4374-ba1a-08cfe45e321c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:43.276659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2762a-d17e-4374-ba1a-08cfe45e321c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:43.277239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2762a-d17e-4374-ba1a-08cfe45e321c None admin] PATH_INFO: `/v3/users/nova` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:43.287628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2762a-d17e-4374-ba1a-08cfe45e321c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:43.289484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2762a-d17e-4374-ba1a-08cfe45e321c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-f1d2762a-d17e-4374-ba1a-08cfe45e321c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:43.308740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:43.311041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:43.311974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:43.311974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:43.311974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:43.311974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:43.311974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:43.311974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:43.311974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:43.311974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:43.311974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:41:43.311974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:43.314840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 321/636] 10.222.0.79 () {56 vars in 1093 bytes} [Fri May 13 13:41:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:43.316526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734442bc-a825-4ae0-ab12-f1a567bbee9f None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:43.317034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734442bc-a825-4ae0-ab12-f1a567bbee9f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:43.317556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734442bc-a825-4ae0-ab12-f1a567bbee9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:43.337138 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-734442bc-a825-4ae0-ab12-f1a567bbee9f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:43.338736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-734442bc-a825-4ae0-ab12-f1a567bbee9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:43.399043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9598e-bb37-4c9f-83b6-3fbabeb9f8b1 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:43.438804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 322/637] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:43.440207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734442bc-a825-4ae0-ab12-f1a567bbee9f None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:43.472775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcde8ed-b329-4069-8db3-d1038a580419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:43.474454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcde8ed-b329-4069-8db3-d1038a580419 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:43.475011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcde8ed-b329-4069-8db3-d1038a580419 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:43.475513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcde8ed-b329-4069-8db3-d1038a580419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:43.561731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcde8ed-b329-4069-8db3-d1038a580419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:43.563630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcde8ed-b329-4069-8db3-d1038a580419 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:43.592096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734442bc-a825-4ae0-ab12-f1a567bbee9f None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:43.628327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0610d-5599-4e8c-bd75-7841013c2442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:43.629110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0610d-5599-4e8c-bd75-7841013c2442 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:43.629317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0610d-5599-4e8c-bd75-7841013c2442 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:43.630100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0610d-5599-4e8c-bd75-7841013c2442 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:43.631031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0610d-5599-4e8c-bd75-7841013c2442 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:43.631648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0610d-5599-4e8c-bd75-7841013c2442 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:43.632804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0610d-5599-4e8c-bd75-7841013c2442 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:43.652773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 323/638] 10.222.0.79 () {56 vars in 1133 bytes} [Fri May 13 13:41:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:43.653793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0610d-5599-4e8c-bd75-7841013c2442 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:43.701556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 316/639] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:43.703068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcde8ed-b329-4069-8db3-d1038a580419 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:43.783969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0610d-5599-4e8c-bd75-7841013c2442 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:43.832331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba76ad5-3390-4f43-b0e6-be321e9657aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:43.833226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba76ad5-3390-4f43-b0e6-be321e9657aa None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:43.834374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba76ad5-3390-4f43-b0e6-be321e9657aa None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:43.834374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba76ad5-3390-4f43-b0e6-be321e9657aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:43.859838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba76ad5-3390-4f43-b0e6-be321e9657aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-cba76ad5-3390-4f43-b0e6-be321e9657aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:43.872771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:43.874315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 324/640] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:43.881924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18099c-0486-4e00-a1bb-e138a96ba790 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:43.883206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18099c-0486-4e00-a1bb-e138a96ba790 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:43.883206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18099c-0486-4e00-a1bb-e138a96ba790 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:43.886612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcde8ed-b329-4069-8db3-d1038a580419 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:43.887900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 325/641] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:43.889478 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18099c-0486-4e00-a1bb-e138a96ba790 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:43.929341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a9640-d243-4d44-a7f6-2f434fa21f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:43.930076 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a9640-d243-4d44-a7f6-2f434fa21f23 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:43.930358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a9640-d243-4d44-a7f6-2f434fa21f23 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:43.930627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a9640-d243-4d44-a7f6-2f434fa21f23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.039807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a9640-d243-4d44-a7f6-2f434fa21f23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:44.042676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a9640-d243-4d44-a7f6-2f434fa21f23 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:44.054786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18099c-0486-4e00-a1bb-e138a96ba790 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:44.096416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb1318-db66-4c44-afd4-3cc81f6d3944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:44.097221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb1318-db66-4c44-afd4-3cc81f6d3944 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.097504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb1318-db66-4c44-afd4-3cc81f6d3944 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.097774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb1318-db66-4c44-afd4-3cc81f6d3944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.109375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb1318-db66-4c44-afd4-3cc81f6d3944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:44.110900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb1318-db66-4c44-afd4-3cc81f6d3944 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-42cb1318-db66-4c44-afd4-3cc81f6d3944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:44.130302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:44.132306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:44.133927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:44.133927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:44.133927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:44.133927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:44.133927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 326/642] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:44.133927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb1318-db66-4c44-afd4-3cc81f6d3944 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:44.171779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 317/643] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:44.173405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a6791-64cb-42fc-9b07-26f1ae9d5802 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.173766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a6791-64cb-42fc-9b07-26f1ae9d5802 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.174122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a6791-64cb-42fc-9b07-26f1ae9d5802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.196011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-4d6a6791-64cb-42fc-9b07-26f1ae9d5802 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:44.197477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4d6a6791-64cb-42fc-9b07-26f1ae9d5802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:44.287837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 318/644] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:44.288719 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a6791-64cb-42fc-9b07-26f1ae9d5802 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:44.301525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb1318-db66-4c44-afd4-3cc81f6d3944 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:44.345948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e83e8-c3a4-40d5-bc2f-b99b05e9c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:44.346625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e83e8-c3a4-40d5-bc2f-b99b05e9c22f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.346952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e83e8-c3a4-40d5-bc2f-b99b05e9c22f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.347225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e83e8-c3a4-40d5-bc2f-b99b05e9c22f None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.347979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e83e8-c3a4-40d5-bc2f-b99b05e9c22f None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:44.348306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e83e8-c3a4-40d5-bc2f-b99b05e9c22f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:44.350212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e83e8-c3a4-40d5-bc2f-b99b05e9c22f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:44.364101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 327/645] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:44.364101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e83e8-c3a4-40d5-bc2f-b99b05e9c22f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:44.434951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a6791-64cb-42fc-9b07-26f1ae9d5802 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:44.477926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e83e8-c3a4-40d5-bc2f-b99b05e9c22f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:44.484741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84766c9-ef09-4d8e-a6dc-5d57f6526393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:44.485706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84766c9-ef09-4d8e-a6dc-5d57f6526393 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.486070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84766c9-ef09-4d8e-a6dc-5d57f6526393 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.486415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84766c9-ef09-4d8e-a6dc-5d57f6526393 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.487278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84766c9-ef09-4d8e-a6dc-5d57f6526393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:44.487788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84766c9-ef09-4d8e-a6dc-5d57f6526393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:44.490002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84766c9-ef09-4d8e-a6dc-5d57f6526393 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:44.502460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 319/646] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:44.503060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84766c9-ef09-4d8e-a6dc-5d57f6526393 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:44.523479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc200ae-f19e-4ac3-b3bf-5a8a424649cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:44.524853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc200ae-f19e-4ac3-b3bf-5a8a424649cc None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.525353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc200ae-f19e-4ac3-b3bf-5a8a424649cc None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.525870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc200ae-f19e-4ac3-b3bf-5a8a424649cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.546342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc200ae-f19e-4ac3-b3bf-5a8a424649cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:44.548622 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc200ae-f19e-4ac3-b3bf-5a8a424649cc None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-0fc200ae-f19e-4ac3-b3bf-5a8a424649cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:44.570921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:44.571873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:44.573018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:44.573018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:44.573018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:44.573018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:44.573018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 328/647] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:44.573418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc200ae-f19e-4ac3-b3bf-5a8a424649cc None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:44.632537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84766c9-ef09-4d8e-a6dc-5d57f6526393 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:44.669949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdf91c-3aaf-447d-904d-fda39aa990b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:44.670752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdf91c-3aaf-447d-904d-fda39aa990b2 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.671048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdf91c-3aaf-447d-904d-fda39aa990b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.671308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdf91c-3aaf-447d-904d-fda39aa990b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.679921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc200ae-f19e-4ac3-b3bf-5a8a424649cc None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:44.681308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdf91c-3aaf-447d-904d-fda39aa990b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-4ecdf91c-3aaf-447d-904d-fda39aa990b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:44.688940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:44.689923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:44.691003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 320/648] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:44.691785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d0e77-7a57-4489-9874-16a5b7988bdc None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.691785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d0e77-7a57-4489-9874-16a5b7988bdc None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.691919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d0e77-7a57-4489-9874-16a5b7988bdc None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.695394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 321/649] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:44.695394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d0e77-7a57-4489-9874-16a5b7988bdc None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:44.714199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6302eff-dde4-4c98-bd2a-f6ef3512cf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:44.715032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6302eff-dde4-4c98-bd2a-f6ef3512cf09 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.715312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6302eff-dde4-4c98-bd2a-f6ef3512cf09 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.715583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6302eff-dde4-4c98-bd2a-f6ef3512cf09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.727033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6302eff-dde4-4c98-bd2a-f6ef3512cf09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:44.728207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6302eff-dde4-4c98-bd2a-f6ef3512cf09 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-c6302eff-dde4-4c98-bd2a-f6ef3512cf09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:44.745074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:44.746246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:44.747177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:44.747177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:44.747177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:44.747177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:44.747177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 329/650] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:44.747177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6302eff-dde4-4c98-bd2a-f6ef3512cf09 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:44.805977 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d0e77-7a57-4489-9874-16a5b7988bdc None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:44.840164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14e14d-51c4-4015-9820-a0c5cd9a419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:44.840882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e14d-51c4-4015-9820-a0c5cd9a419b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.841154 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e14d-51c4-4015-9820-a0c5cd9a419b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.841446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e14d-51c4-4015-9820-a0c5cd9a419b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.849233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14e14d-51c4-4015-9820-a0c5cd9a419b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:44.850316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6302eff-dde4-4c98-bd2a-f6ef3512cf09 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:44.850632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14e14d-51c4-4015-9820-a0c5cd9a419b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-4d14e14d-51c4-4015-9820-a0c5cd9a419b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:44.867567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:44.868577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:44.869468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:44.869468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:44.869468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:44.869468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:44.869468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:44.869468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:44.869468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:44.869468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:44.869468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 322/651] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:44.870429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14e14d-51c4-4015-9820-a0c5cd9a419b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:44.887609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4eb56-7355-44c1-aea8-810c27a6c4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:44.887609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4eb56-7355-44c1-aea8-810c27a6c4cb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.887609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4eb56-7355-44c1-aea8-810c27a6c4cb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.888006 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4eb56-7355-44c1-aea8-810c27a6c4cb None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.889046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4eb56-7355-44c1-aea8-810c27a6c4cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:44.889046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4eb56-7355-44c1-aea8-810c27a6c4cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:44.890203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4eb56-7355-44c1-aea8-810c27a6c4cb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:44.904245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 330/652] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:44.905578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92970944-3627-4f8d-b873-818b9a8a9cee None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:44.905858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92970944-3627-4f8d-b873-818b9a8a9cee None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:44.906134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92970944-3627-4f8d-b873-818b9a8a9cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:44.924329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-92970944-3627-4f8d-b873-818b9a8a9cee None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:44.925543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-92970944-3627-4f8d-b873-818b9a8a9cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:44.989689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14e14d-51c4-4015-9820-a0c5cd9a419b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:45.004193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 331/653] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:45.004859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92970944-3627-4f8d-b873-818b9a8a9cee None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:45.032982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e8bf1-dcd7-41e0-9b44-ac7b5258a074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:45.033690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8bf1-dcd7-41e0-9b44-ac7b5258a074 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:45.034058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8bf1-dcd7-41e0-9b44-ac7b5258a074 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:45.034342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e8bf1-dcd7-41e0-9b44-ac7b5258a074 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:45.035197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e8bf1-dcd7-41e0-9b44-ac7b5258a074 None admin] RBAC: Adding query filter params (name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:45.035498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e8bf1-dcd7-41e0-9b44-ac7b5258a074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:45.037552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e8bf1-dcd7-41e0-9b44-ac7b5258a074 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:45.048449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 323/654] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:45.048449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e8bf1-dcd7-41e0-9b44-ac7b5258a074 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:45.126337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92970944-3627-4f8d-b873-818b9a8a9cee None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:45.176169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3ed75-bd64-4950-8440-89d4708009df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:45.176925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3ed75-bd64-4950-8440-89d4708009df None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:45.177199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3ed75-bd64-4950-8440-89d4708009df None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:45.177467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3ed75-bd64-4950-8440-89d4708009df None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:45.178209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3ed75-bd64-4950-8440-89d4708009df None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:45.178508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3ed75-bd64-4950-8440-89d4708009df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:45.179955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3ed75-bd64-4950-8440-89d4708009df None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:45.189668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e8bf1-dcd7-41e0-9b44-ac7b5258a074 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:45.200032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 332/655] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:45.201171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3ed75-bd64-4950-8440-89d4708009df None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:45.233496 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bcd7f-340d-44fc-bec4-68cce487ca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:45.234183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bcd7f-340d-44fc-bec4-68cce487ca46 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:45.234461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bcd7f-340d-44fc-bec4-68cce487ca46 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:45.234764 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bcd7f-340d-44fc-bec4-68cce487ca46 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:45.235562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bcd7f-340d-44fc-bec4-68cce487ca46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:45.235865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bcd7f-340d-44fc-bec4-68cce487ca46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:45.237355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bcd7f-340d-44fc-bec4-68cce487ca46 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:45.250348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 324/656] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:45.251243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bcd7f-340d-44fc-bec4-68cce487ca46 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:45.315807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3ed75-bd64-4950-8440-89d4708009df None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:45.358585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a90d0-e3c8-422d-80e3-36afdfe5be8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:45.359885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a90d0-e3c8-422d-80e3-36afdfe5be8f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:45.360273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a90d0-e3c8-422d-80e3-36afdfe5be8f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:45.360718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a90d0-e3c8-422d-80e3-36afdfe5be8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:45.372113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a90d0-e3c8-422d-80e3-36afdfe5be8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:45.374605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a90d0-e3c8-422d-80e3-36afdfe5be8f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:45.388878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bcd7f-340d-44fc-bec4-68cce487ca46 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-415a90d0-e3c8-422d-80e3-36afdfe5be8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:45.398908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:45.399994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:45.400970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:45.400970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:45.400970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:45.400970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:45.400970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:45.400970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:45.400970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:45.400970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:45.400970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 333/657] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:45.402230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a90d0-e3c8-422d-80e3-36afdfe5be8f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:45.441538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246264d-4d1c-4428-b89e-f88eb6164462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:45.442788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246264d-4d1c-4428-b89e-f88eb6164462 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:45.443301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246264d-4d1c-4428-b89e-f88eb6164462 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:45.443696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246264d-4d1c-4428-b89e-f88eb6164462 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:45.461319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246264d-4d1c-4428-b89e-f88eb6164462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-b246264d-4d1c-4428-b89e-f88eb6164462 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:45.478226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:45.481082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 325/658] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:45.483467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246264d-4d1c-4428-b89e-f88eb6164462 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:45.556175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a90d0-e3c8-422d-80e3-36afdfe5be8f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:45.596236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db635d-e0f3-4327-8edd-45f8d4c56d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:45.597051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db635d-e0f3-4327-8edd-45f8d4c56d03 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:45.597330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db635d-e0f3-4327-8edd-45f8d4c56d03 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:45.597831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db635d-e0f3-4327-8edd-45f8d4c56d03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:45.608813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db635d-e0f3-4327-8edd-45f8d4c56d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:45.610279 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db635d-e0f3-4327-8edd-45f8d4c56d03 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:45.616564 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246264d-4d1c-4428-b89e-f88eb6164462 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-00db635d-e0f3-4327-8edd-45f8d4c56d03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:45.628760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:45.629895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:45.630818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:45.630818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:45.630818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:45.630818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:45.630818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 334/659] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:45.630818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db635d-e0f3-4327-8edd-45f8d4c56d03 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:45.667292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8a6ad-4ae2-40bc-ac6e-fc37ec3c9742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:45.668008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8a6ad-4ae2-40bc-ac6e-fc37ec3c9742 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:45.668284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8a6ad-4ae2-40bc-ac6e-fc37ec3c9742 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:45.668554 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8a6ad-4ae2-40bc-ac6e-fc37ec3c9742 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:45.677415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8a6ad-4ae2-40bc-ac6e-fc37ec3c9742 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:45.678914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8a6ad-4ae2-40bc-ac6e-fc37ec3c9742 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-e9d8a6ad-4ae2-40bc-ac6e-fc37ec3c9742 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:45.694837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:45.696845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:45.698771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:45.698771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:45.698771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:45.698771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:45.698771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:45.698771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:45.698771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:45.698771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:45.698771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:41:45.698771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:45.700911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 326/660] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:45.701841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8a6ad-4ae2-40bc-ac6e-fc37ec3c9742 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:45.752959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db635d-e0f3-4327-8edd-45f8d4c56d03 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:45.798515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf4176-afb6-41c3-bb81-06ce0f28e2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:45.799196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf4176-afb6-41c3-bb81-06ce0f28e2f6 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:45.799478 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf4176-afb6-41c3-bb81-06ce0f28e2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:45.799749 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf4176-afb6-41c3-bb81-06ce0f28e2f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:45.800583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf4176-afb6-41c3-bb81-06ce0f28e2f6 None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=cc4f45e859d84ceaa5a01477e83e098e) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:45.800890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf4176-afb6-41c3-bb81-06ce0f28e2f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:45.802375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf4176-afb6-41c3-bb81-06ce0f28e2f6 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:45.809145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 335/661] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:45 2022] GET /identity/v3/role_assignments?user.id=cc4f45e859d84ceaa5a01477e83e098e&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:45.809897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf4176-afb6-41c3-bb81-06ce0f28e2f6 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:45.828178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8a6ad-4ae2-40bc-ac6e-fc37ec3c9742 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:45.875833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfb6ba-85a4-4163-b7f5-6df13ff90650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:45.876535 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfb6ba-85a4-4163-b7f5-6df13ff90650 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:45.876920 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfb6ba-85a4-4163-b7f5-6df13ff90650 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:45.877214 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfb6ba-85a4-4163-b7f5-6df13ff90650 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:45.878538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfb6ba-85a4-4163-b7f5-6df13ff90650 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:45.879010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfb6ba-85a4-4163-b7f5-6df13ff90650 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:45.881442 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfb6ba-85a4-4163-b7f5-6df13ff90650 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:45.898732 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 327/662] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:45.900091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfb6ba-85a4-4163-b7f5-6df13ff90650 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:45.987018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf4176-afb6-41c3-bb81-06ce0f28e2f6 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:46.033191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738de365-c68f-4f4c-b75e-387db40fe44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:46.033945 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738de365-c68f-4f4c-b75e-387db40fe44d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:46.034218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738de365-c68f-4f4c-b75e-387db40fe44d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:46.034572 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738de365-c68f-4f4c-b75e-387db40fe44d None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:46.035365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738de365-c68f-4f4c-b75e-387db40fe44d None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:46.035699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738de365-c68f-4f4c-b75e-387db40fe44d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:46.038151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738de365-c68f-4f4c-b75e-387db40fe44d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:46.048178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 336/663] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:46.049083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738de365-c68f-4f4c-b75e-387db40fe44d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:46.051130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfb6ba-85a4-4163-b7f5-6df13ff90650 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:46.106379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3d15f-51fb-4d7c-afe4-ac692a7dc52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:46.107092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3d15f-51fb-4d7c-afe4-ac692a7dc52f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:46.107365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3d15f-51fb-4d7c-afe4-ac692a7dc52f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:46.107638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3d15f-51fb-4d7c-afe4-ac692a7dc52f None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:46.108371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3d15f-51fb-4d7c-afe4-ac692a7dc52f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:46.108665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3d15f-51fb-4d7c-afe4-ac692a7dc52f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:46.110109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3d15f-51fb-4d7c-afe4-ac692a7dc52f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:46.128121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 328/664] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:46.128121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3d15f-51fb-4d7c-afe4-ac692a7dc52f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:46.199459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738de365-c68f-4f4c-b75e-387db40fe44d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:46.246777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4329e-c80b-4759-9905-287c61cd0948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:46.248245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4329e-c80b-4759-9905-287c61cd0948 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:46.248779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4329e-c80b-4759-9905-287c61cd0948 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:46.249265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4329e-c80b-4759-9905-287c61cd0948 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:46.250318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4329e-c80b-4759-9905-287c61cd0948 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:46.252514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4329e-c80b-4759-9905-287c61cd0948 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:46.256817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4329e-c80b-4759-9905-287c61cd0948 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:46.273134 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3d15f-51fb-4d7c-afe4-ac692a7dc52f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:46.286741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 337/665] 10.222.0.79 () {56 vars in 1137 bytes} [Fri May 13 13:41:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:46.287661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4329e-c80b-4759-9905-287c61cd0948 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:46.330672 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f105207-1213-4c6e-9c5f-9a2588e9b89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:46.331887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105207-1213-4c6e-9c5f-9a2588e9b89d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:46.332354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105207-1213-4c6e-9c5f-9a2588e9b89d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:46.333338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f105207-1213-4c6e-9c5f-9a2588e9b89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:46.334913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f105207-1213-4c6e-9c5f-9a2588e9b89d None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=24ad0dd9d265425087be2be8d245b7fc) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:46.335449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f105207-1213-4c6e-9c5f-9a2588e9b89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:46.338090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f105207-1213-4c6e-9c5f-9a2588e9b89d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:46.359071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 329/666] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:46 2022] GET /identity/v3/role_assignments?user.id=24ad0dd9d265425087be2be8d245b7fc&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:46.359071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f105207-1213-4c6e-9c5f-9a2588e9b89d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:46.413080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4329e-c80b-4759-9905-287c61cd0948 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:46.463697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c39ac-1923-46c9-82f0-313122c5031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:46.464716 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c39ac-1923-46c9-82f0-313122c5031d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:46.465204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c39ac-1923-46c9-82f0-313122c5031d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:46.465700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c39ac-1923-46c9-82f0-313122c5031d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:46.483179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c39ac-1923-46c9-82f0-313122c5031d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:46.486047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c39ac-1923-46c9-82f0-313122c5031d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:46.516011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f105207-1213-4c6e-9c5f-9a2588e9b89d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-e87c39ac-1923-46c9-82f0-313122c5031d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:46.516954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:46.518566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:46.521230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:46.521230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:46.521230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:46.521230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:46.527599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 338/667] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:46.529004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c39ac-1923-46c9-82f0-313122c5031d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:46.565095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cf987-6793-492a-9030-f0fe58ee8e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:46.566103 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cf987-6793-492a-9030-f0fe58ee8e4f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:46.566495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cf987-6793-492a-9030-f0fe58ee8e4f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:46.567468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cf987-6793-492a-9030-f0fe58ee8e4f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:46.576881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cf987-6793-492a-9030-f0fe58ee8e4f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:46.578616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cf987-6793-492a-9030-f0fe58ee8e4f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-869cf987-6793-492a-9030-f0fe58ee8e4f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:46.595391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:46.597385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:46.604365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 330/668] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:46.606495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cf987-6793-492a-9030-f0fe58ee8e4f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:46.693626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c39ac-1923-46c9-82f0-313122c5031d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:46.748128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986eeac9-6219-448a-accc-e93f756ba344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:46.748805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986eeac9-6219-448a-accc-e93f756ba344 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:46.749087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986eeac9-6219-448a-accc-e93f756ba344 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:46.749379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986eeac9-6219-448a-accc-e93f756ba344 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:46.773198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cf987-6793-492a-9030-f0fe58ee8e4f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:46.848940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6234162-c1e3-4f74-aa91-50898d2114cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:46.850140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6234162-c1e3-4f74-aa91-50898d2114cc None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:46.850643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6234162-c1e3-4f74-aa91-50898d2114cc None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:46.851171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6234162-c1e3-4f74-aa91-50898d2114cc None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:46.852515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6234162-c1e3-4f74-aa91-50898d2114cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:46.853086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6234162-c1e3-4f74-aa91-50898d2114cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:46.855813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6234162-c1e3-4f74-aa91-50898d2114cc None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:46.863030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986eeac9-6219-448a-accc-e93f756ba344 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:46.866019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986eeac9-6219-448a-accc-e93f756ba344 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:46.881642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 331/669] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:46.890450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6234162-c1e3-4f74-aa91-50898d2114cc None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:47.017948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 339/670] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:47.020164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986eeac9-6219-448a-accc-e93f756ba344 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:47.094329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6234162-c1e3-4f74-aa91-50898d2114cc None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:47.157587 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038d65a-459b-4017-bdcd-3d8f50ff9014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:47.158015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038d65a-459b-4017-bdcd-3d8f50ff9014 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:47.158441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038d65a-459b-4017-bdcd-3d8f50ff9014 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:47.158911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038d65a-459b-4017-bdcd-3d8f50ff9014 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:47.160330 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038d65a-459b-4017-bdcd-3d8f50ff9014 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:47.160853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038d65a-459b-4017-bdcd-3d8f50ff9014 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:47.166264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038d65a-459b-4017-bdcd-3d8f50ff9014 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:47.198803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 332/671] 10.222.0.79 () {56 vars in 1139 bytes} [Fri May 13 13:41:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:47.198803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038d65a-459b-4017-bdcd-3d8f50ff9014 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:47.224928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986eeac9-6219-448a-accc-e93f756ba344 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:47.295117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc9863-8299-4cdc-b517-1f2c0dcc55e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:47.297126 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc9863-8299-4cdc-b517-1f2c0dcc55e7 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:47.297898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc9863-8299-4cdc-b517-1f2c0dcc55e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:47.299158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc9863-8299-4cdc-b517-1f2c0dcc55e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:47.309370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc9863-8299-4cdc-b517-1f2c0dcc55e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:47.311087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc9863-8299-4cdc-b517-1f2c0dcc55e7 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-29fc9863-8299-4cdc-b517-1f2c0dcc55e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:47.329072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:47.330358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:47.331341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 340/672] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:41:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:47.331955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc9863-8299-4cdc-b517-1f2c0dcc55e7 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:47.358472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038d65a-459b-4017-bdcd-3d8f50ff9014 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:47.417712 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388377d9-8f97-4a86-8962-dc947ae730e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:47.419313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388377d9-8f97-4a86-8962-dc947ae730e0 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:47.419846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388377d9-8f97-4a86-8962-dc947ae730e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:47.420329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388377d9-8f97-4a86-8962-dc947ae730e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:47.434481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388377d9-8f97-4a86-8962-dc947ae730e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:47.436422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388377d9-8f97-4a86-8962-dc947ae730e0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-388377d9-8f97-4a86-8962-dc947ae730e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:47.468765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:47.469814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:47.471725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:47.471725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:47.471725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:47.471725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:47.475987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 333/673] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:47.477930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388377d9-8f97-4a86-8962-dc947ae730e0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:47.500785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc9863-8299-4cdc-b517-1f2c0dcc55e7 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:47.556594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8c7c3-a257-4de5-9382-723f850f094c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:47.557503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8c7c3-a257-4de5-9382-723f850f094c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:47.557984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8c7c3-a257-4de5-9382-723f850f094c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:47.558468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8c7c3-a257-4de5-9382-723f850f094c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:47.624545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388377d9-8f97-4a86-8962-dc947ae730e0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:47.652598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8c7c3-a257-4de5-9382-723f850f094c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:47.654101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8c7c3-a257-4de5-9382-723f850f094c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:47.667631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da130dbe-6bc5-4645-ae1a-02795f90ee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:47.668780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da130dbe-6bc5-4645-ae1a-02795f90ee3e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:47.669369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da130dbe-6bc5-4645-ae1a-02795f90ee3e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:47.670022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da130dbe-6bc5-4645-ae1a-02795f90ee3e None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:47.671556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da130dbe-6bc5-4645-ae1a-02795f90ee3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:47.672336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da130dbe-6bc5-4645-ae1a-02795f90ee3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:47.674407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da130dbe-6bc5-4645-ae1a-02795f90ee3e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:47.698760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 334/674] 10.222.0.79 () {56 vars in 1135 bytes} [Fri May 13 13:41:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:47.700812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da130dbe-6bc5-4645-ae1a-02795f90ee3e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:47.814765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 341/675] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:47.814765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8c7c3-a257-4de5-9382-723f850f094c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:47.854241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da130dbe-6bc5-4645-ae1a-02795f90ee3e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:47.897931 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd50d4-547a-4c94-bf91-2efa5703e9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:47.899144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd50d4-547a-4c94-bf91-2efa5703e9c8 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:47.899643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd50d4-547a-4c94-bf91-2efa5703e9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:47.900252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd50d4-547a-4c94-bf91-2efa5703e9c8 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:47.901452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd50d4-547a-4c94-bf91-2efa5703e9c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:47.901988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd50d4-547a-4c94-bf91-2efa5703e9c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:47.903976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd50d4-547a-4c94-bf91-2efa5703e9c8 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:47.923128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 335/676] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:47.924534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd50d4-547a-4c94-bf91-2efa5703e9c8 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:47.968941 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8c7c3-a257-4de5-9382-723f850f094c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:48.012378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececba63-e196-42c2-829f-87bc4043d0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:48.013047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececba63-e196-42c2-829f-87bc4043d0cb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:48.013327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececba63-e196-42c2-829f-87bc4043d0cb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:48.013635 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececba63-e196-42c2-829f-87bc4043d0cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:48.105657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececba63-e196-42c2-829f-87bc4043d0cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:48.107482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececba63-e196-42c2-829f-87bc4043d0cb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:48.116891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd50d4-547a-4c94-bf91-2efa5703e9c8 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:48.182525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b5f06-f43e-4a59-bad5-321db74bed82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:48.184569 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b5f06-f43e-4a59-bad5-321db74bed82 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:48.185378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b5f06-f43e-4a59-bad5-321db74bed82 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:48.186184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b5f06-f43e-4a59-bad5-321db74bed82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:48.207999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b5f06-f43e-4a59-bad5-321db74bed82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:48.211976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b5f06-f43e-4a59-bad5-321db74bed82 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:48.226038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 342/677] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:48.226978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececba63-e196-42c2-829f-87bc4043d0cb None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-b93b5f06-f43e-4a59-bad5-321db74bed82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:48.248027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:48.249472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:48.251072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:48.251072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:48.251072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:48.251072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:48.259218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 336/678] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:48.261254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b5f06-f43e-4a59-bad5-321db74bed82 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:48.346052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececba63-e196-42c2-829f-87bc4043d0cb None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:48.378756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b5f06-f43e-4a59-bad5-321db74bed82 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:48.401429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e364476-9fe5-4fe0-9462-ddc94471d22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:48.403000 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e364476-9fe5-4fe0-9462-ddc94471d22a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:48.403508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e364476-9fe5-4fe0-9462-ddc94471d22a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:48.404003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e364476-9fe5-4fe0-9462-ddc94471d22a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:48.418437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e364476-9fe5-4fe0-9462-ddc94471d22a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:48.421483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e364476-9fe5-4fe0-9462-ddc94471d22a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:48.427124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d72d5f-9d66-4146-a4e1-b748628ce917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:48.428356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72d5f-9d66-4146-a4e1-b748628ce917 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:48.428850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72d5f-9d66-4146-a4e1-b748628ce917 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:48.429349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d72d5f-9d66-4146-a4e1-b748628ce917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:48.443470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d72d5f-9d66-4146-a4e1-b748628ce917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-5e364476-9fe5-4fe0-9462-ddc94471d22a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:48.444181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:48.445067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:48.445962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:48.445962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:48.445962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:48.445962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:48.445962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:48.445962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:48.445962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:48.445962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:48.446382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 343/679] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:48.447290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e364476-9fe5-4fe0-9462-ddc94471d22a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:48.448687 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d72d5f-9d66-4146-a4e1-b748628ce917 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-43d72d5f-9d66-4146-a4e1-b748628ce917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:48.468860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:48.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:48.471441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:48.471441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:48.471441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:48.471441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:48.473582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 337/680] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:48.474804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d72d5f-9d66-4146-a4e1-b748628ce917 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:48.608638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d72d5f-9d66-4146-a4e1-b748628ce917 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:48.630068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e364476-9fe5-4fe0-9462-ddc94471d22a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:48.650650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de2bd9-0e03-485d-b823-761d3c318076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:48.652212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de2bd9-0e03-485d-b823-761d3c318076 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:48.652737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de2bd9-0e03-485d-b823-761d3c318076 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:48.653221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de2bd9-0e03-485d-b823-761d3c318076 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:48.654346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de2bd9-0e03-485d-b823-761d3c318076 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:48.654943 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de2bd9-0e03-485d-b823-761d3c318076 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:48.656764 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de2bd9-0e03-485d-b823-761d3c318076 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:48.680667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 338/681] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:48.680667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de2bd9-0e03-485d-b823-761d3c318076 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:48.695407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8fd2c-e9c1-4391-92df-a940004b5692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:48.696026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8fd2c-e9c1-4391-92df-a940004b5692 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:48.696307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8fd2c-e9c1-4391-92df-a940004b5692 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:48.696579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8fd2c-e9c1-4391-92df-a940004b5692 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:48.697367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8fd2c-e9c1-4391-92df-a940004b5692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:48.697747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8fd2c-e9c1-4391-92df-a940004b5692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:48.699197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8fd2c-e9c1-4391-92df-a940004b5692 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:48.730658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 344/682] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:48.730658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9220c78-95e7-4287-b4a7-9100122836dd None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:48.730658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9220c78-95e7-4287-b4a7-9100122836dd None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:48.730658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9220c78-95e7-4287-b4a7-9100122836dd None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:48.730658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 345/683] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:48.730658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9220c78-95e7-4287-b4a7-9100122836dd None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:48.860004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de2bd9-0e03-485d-b823-761d3c318076 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:48.907141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9220c78-95e7-4287-b4a7-9100122836dd None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:48.923002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a97cf-dde6-4d9e-851e-0949e0f7b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:48.923687 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a97cf-dde6-4d9e-851e-0949e0f7b0ce None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:48.923982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a97cf-dde6-4d9e-851e-0949e0f7b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:48.924289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a97cf-dde6-4d9e-851e-0949e0f7b0ce None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:48.925005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a97cf-dde6-4d9e-851e-0949e0f7b0ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:48.925312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a97cf-dde6-4d9e-851e-0949e0f7b0ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:48.927518 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a97cf-dde6-4d9e-851e-0949e0f7b0ce None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:48.939766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 339/684] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:48.940647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a97cf-dde6-4d9e-851e-0949e0f7b0ce None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:48.969258 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed6d35-fcc3-4990-b31a-a3e17150cf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:48.969953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed6d35-fcc3-4990-b31a-a3e17150cf44 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:48.970240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed6d35-fcc3-4990-b31a-a3e17150cf44 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:48.970513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed6d35-fcc3-4990-b31a-a3e17150cf44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:48.974880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed6d35-fcc3-4990-b31a-a3e17150cf44 None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=e03fa33a68ff4567bc25c9611cd9ae8b) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:48.974880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed6d35-fcc3-4990-b31a-a3e17150cf44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:48.974880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed6d35-fcc3-4990-b31a-a3e17150cf44 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:48.991114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 346/685] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:48 2022] GET /identity/v3/role_assignments?user.id=e03fa33a68ff4567bc25c9611cd9ae8b&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:48.991114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6bf04-e656-4108-adc5-c0353223946a None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:48.991114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6bf04-e656-4108-adc5-c0353223946a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:48.991114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6bf04-e656-4108-adc5-c0353223946a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:49.010625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-d9b6bf04-e656-4108-adc5-c0353223946a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:49.012093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d9b6bf04-e656-4108-adc5-c0353223946a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:49.105618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a97cf-dde6-4d9e-851e-0949e0f7b0ce None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:49.141832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 347/686] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:49.141832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6bf04-e656-4108-adc5-c0353223946a None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:49.207018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad489-f2e8-4b1f-9f60-80803ce2c54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:49.210422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad489-f2e8-4b1f-9f60-80803ce2c54e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:49.211385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad489-f2e8-4b1f-9f60-80803ce2c54e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:49.212648 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aad489-f2e8-4b1f-9f60-80803ce2c54e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:49.234135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aad489-f2e8-4b1f-9f60-80803ce2c54e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:49.237529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aad489-f2e8-4b1f-9f60-80803ce2c54e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-c4aad489-f2e8-4b1f-9f60-80803ce2c54e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:49.276271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:49.277433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:49.280049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:49.280049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:49.280049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:49.280049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:49.280049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:49.280049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:49.280049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:49.280049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:49.281303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 340/687] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:49.282343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad489-f2e8-4b1f-9f60-80803ce2c54e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:49.375113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6bf04-e656-4108-adc5-c0353223946a None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:49.427130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eb553-b4ab-41ae-b12c-e9a0b2b683fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:49.427963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eb553-b4ab-41ae-b12c-e9a0b2b683fc None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:49.428249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eb553-b4ab-41ae-b12c-e9a0b2b683fc None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:49.428528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eb553-b4ab-41ae-b12c-e9a0b2b683fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:49.440316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468eb553-b4ab-41ae-b12c-e9a0b2b683fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:49.443058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468eb553-b4ab-41ae-b12c-e9a0b2b683fc None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:49.464521 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aad489-f2e8-4b1f-9f60-80803ce2c54e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-468eb553-b4ab-41ae-b12c-e9a0b2b683fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:49.471361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:49.474494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:49.477322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:49.477322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:49.477322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:49.477322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:49.477322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:49.477322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:49.477322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:49.477322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:49.477322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 348/688] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:49.477322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eb553-b4ab-41ae-b12c-e9a0b2b683fc None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:49.518261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ef8d4-83a8-4814-8119-27390d45c0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:49.519147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ef8d4-83a8-4814-8119-27390d45c0b1 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:49.519427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ef8d4-83a8-4814-8119-27390d45c0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:49.519755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ef8d4-83a8-4814-8119-27390d45c0b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:49.537509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ef8d4-83a8-4814-8119-27390d45c0b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:49.539110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ef8d4-83a8-4814-8119-27390d45c0b1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-d54ef8d4-83a8-4814-8119-27390d45c0b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:49.560975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:49.561978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:49.564400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:49.564400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:49.564400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:49.564400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:49.564400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 341/689] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:49.564400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec85be9-0805-457a-9d6f-349e8f2fa847 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:49.564918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec85be9-0805-457a-9d6f-349e8f2fa847 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:49.565045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec85be9-0805-457a-9d6f-349e8f2fa847 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:49.569841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 342/690] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:49.570109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec85be9-0805-457a-9d6f-349e8f2fa847 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:49.685729 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eb553-b4ab-41ae-b12c-e9a0b2b683fc None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:49.733682 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec85be9-0805-457a-9d6f-349e8f2fa847 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:49.744699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff9a33-9927-4372-8f64-f4ed725450d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:49.745775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff9a33-9927-4372-8f64-f4ed725450d3 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:49.746159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff9a33-9927-4372-8f64-f4ed725450d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:49.746438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff9a33-9927-4372-8f64-f4ed725450d3 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:49.747488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff9a33-9927-4372-8f64-f4ed725450d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:49.747827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff9a33-9927-4372-8f64-f4ed725450d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:49.749654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff9a33-9927-4372-8f64-f4ed725450d3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:49.768580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 349/691] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:49.770591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85b68-209c-44ef-86eb-daadec67837d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:49.771053 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85b68-209c-44ef-86eb-daadec67837d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:49.771474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e85b68-209c-44ef-86eb-daadec67837d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:49.799652 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bad291-b9cc-454b-a69c-1086732705b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:49.800657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-c2e85b68-209c-44ef-86eb-daadec67837d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:49.800830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bad291-b9cc-454b-a69c-1086732705b9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:49.801735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bad291-b9cc-454b-a69c-1086732705b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:49.802189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bad291-b9cc-454b-a69c-1086732705b9 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:49.803264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c2e85b68-209c-44ef-86eb-daadec67837d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:49.803626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bad291-b9cc-454b-a69c-1086732705b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:49.804119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bad291-b9cc-454b-a69c-1086732705b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:49.806760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bad291-b9cc-454b-a69c-1086732705b9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:49.826581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 343/692] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:49.828109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bad291-b9cc-454b-a69c-1086732705b9 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:49.930786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 350/693] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:49.932317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e85b68-209c-44ef-86eb-daadec67837d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:50.040828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bad291-b9cc-454b-a69c-1086732705b9 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:50.091278 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e85b68-209c-44ef-86eb-daadec67837d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:50.093325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fba9e8-cc0c-4797-908f-0a1e1bdc2390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:50.095277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fba9e8-cc0c-4797-908f-0a1e1bdc2390 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:50.095688 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fba9e8-cc0c-4797-908f-0a1e1bdc2390 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:50.095967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fba9e8-cc0c-4797-908f-0a1e1bdc2390 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:50.096929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fba9e8-cc0c-4797-908f-0a1e1bdc2390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:50.097234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fba9e8-cc0c-4797-908f-0a1e1bdc2390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:50.099781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fba9e8-cc0c-4797-908f-0a1e1bdc2390 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:50.131742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 344/694] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:50.133964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fba9e8-cc0c-4797-908f-0a1e1bdc2390 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:50.158379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fc788-9beb-4db7-a421-9adb085e91a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:50.161175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fc788-9beb-4db7-a421-9adb085e91a2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:50.162252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fc788-9beb-4db7-a421-9adb085e91a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:50.162527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fc788-9beb-4db7-a421-9adb085e91a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:50.163726 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fc788-9beb-4db7-a421-9adb085e91a2 None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=8f784cc461ab48ad92140680758a3704) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:50.164060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fc788-9beb-4db7-a421-9adb085e91a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:50.166083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fc788-9beb-4db7-a421-9adb085e91a2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:50.186981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 351/695] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:49 2022] GET /identity/v3/role_assignments?user.id=8f784cc461ab48ad92140680758a3704&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:50.186981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fc788-9beb-4db7-a421-9adb085e91a2 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:50.353898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fba9e8-cc0c-4797-908f-0a1e1bdc2390 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:50.354376 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fc788-9beb-4db7-a421-9adb085e91a2 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:50.402827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c37e6-8d58-4d10-9414-e600aed9bd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:50.404293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c37e6-8d58-4d10-9414-e600aed9bd74 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:50.405102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c37e6-8d58-4d10-9414-e600aed9bd74 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:50.405576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c37e6-8d58-4d10-9414-e600aed9bd74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:50.406698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c37e6-8d58-4d10-9414-e600aed9bd74 None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=4139cfe906bd4c7d94226ab589577627) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:50.407279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c37e6-8d58-4d10-9414-e600aed9bd74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:50.408274 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efea2f-fa95-42c5-ade2-1099b949cc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:50.409525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c37e6-8d58-4d10-9414-e600aed9bd74 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:50.410130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efea2f-fa95-42c5-ade2-1099b949cc7a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:50.411462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efea2f-fa95-42c5-ade2-1099b949cc7a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:50.412201 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efea2f-fa95-42c5-ade2-1099b949cc7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:50.426978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 345/696] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:50 2022] GET /identity/v3/role_assignments?user.id=4139cfe906bd4c7d94226ab589577627&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:50.426978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c37e6-8d58-4d10-9414-e600aed9bd74 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:50.442211 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efea2f-fa95-42c5-ade2-1099b949cc7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-68efea2f-fa95-42c5-ade2-1099b949cc7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:50.473566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:50.475884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:50.477783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:50.484170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 352/697] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:41:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:50.492043 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efea2f-fa95-42c5-ade2-1099b949cc7a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:50.599001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c37e6-8d58-4d10-9414-e600aed9bd74 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:50.679823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307622ad-5696-4b45-b6ae-d7a0d22f30d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:50.681161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307622ad-5696-4b45-b6ae-d7a0d22f30d6 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:50.681791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307622ad-5696-4b45-b6ae-d7a0d22f30d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:50.682429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307622ad-5696-4b45-b6ae-d7a0d22f30d6 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:50.684070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307622ad-5696-4b45-b6ae-d7a0d22f30d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:50.687206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307622ad-5696-4b45-b6ae-d7a0d22f30d6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:50.710672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efea2f-fa95-42c5-ade2-1099b949cc7a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:50.741932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 346/698] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:41:50 2022] POST /identity/v3/users => generated 254 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:41:50.796306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f21e5-7944-4c68-a202-0957af6b80d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:50.797020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f21e5-7944-4c68-a202-0957af6b80d5 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:50.797308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f21e5-7944-4c68-a202-0957af6b80d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:50.797586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f21e5-7944-4c68-a202-0957af6b80d5 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:50.798373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f21e5-7944-4c68-a202-0957af6b80d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:50.798684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f21e5-7944-4c68-a202-0957af6b80d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:50.800600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f21e5-7944-4c68-a202-0957af6b80d5 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:50.822533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 353/699] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:50.832094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307622ad-5696-4b45-b6ae-d7a0d22f30d6 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:51.038670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307622ad-5696-4b45-b6ae-d7a0d22f30d6 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:51.090810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bb09a-3c88-49d7-98b5-6f82fb7c0f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:51.092035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bb09a-3c88-49d7-98b5-6f82fb7c0f01 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:51.092626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bb09a-3c88-49d7-98b5-6f82fb7c0f01 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:51.093470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bb09a-3c88-49d7-98b5-6f82fb7c0f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:51.109353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bb09a-3c88-49d7-98b5-6f82fb7c0f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:51.111503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bb09a-3c88-49d7-98b5-6f82fb7c0f01 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-fd7bb09a-3c88-49d7-98b5-6f82fb7c0f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:51.138927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:51.140373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:51.141903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:51.141903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:51.141903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:51.141903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:51.144185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 347/700] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:51.150298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f21e5-7944-4c68-a202-0957af6b80d5 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:51.326771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f21e5-7944-4c68-a202-0957af6b80d5 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:51.374289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c7a67-9b9f-4114-a691-fd4e563a0df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:51.374996 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c7a67-9b9f-4114-a691-fd4e563a0df9 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:51.375284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c7a67-9b9f-4114-a691-fd4e563a0df9 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:51.375561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c7a67-9b9f-4114-a691-fd4e563a0df9 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:51.376328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c7a67-9b9f-4114-a691-fd4e563a0df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:51.376754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c7a67-9b9f-4114-a691-fd4e563a0df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:51.378330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c7a67-9b9f-4114-a691-fd4e563a0df9 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:51.403264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 354/701] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:51.409566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bb09a-3c88-49d7-98b5-6f82fb7c0f01 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:51.566123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bb09a-3c88-49d7-98b5-6f82fb7c0f01 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:51.624884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23894a8-13ce-47e7-8239-b457b5235cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:51.626073 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23894a8-13ce-47e7-8239-b457b5235cdb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:51.626627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23894a8-13ce-47e7-8239-b457b5235cdb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:51.627288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23894a8-13ce-47e7-8239-b457b5235cdb None admin] PATH_INFO: `/v3/users/nova` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:51.637344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23894a8-13ce-47e7-8239-b457b5235cdb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:51.643866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23894a8-13ce-47e7-8239-b457b5235cdb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-c23894a8-13ce-47e7-8239-b457b5235cdb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:51.667234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:51.669028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:51.670830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:51.670830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:51.670830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:51.670830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:51.670830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:51.670830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:51.670830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:51.670830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:51.670830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:41:51.670830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:51.674431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 348/702] 10.222.0.79 () {56 vars in 1093 bytes} [Fri May 13 13:41:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:51.680649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c7a67-9b9f-4114-a691-fd4e563a0df9 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:51.851913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c7a67-9b9f-4114-a691-fd4e563a0df9 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:51.894698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56885b-1c59-490b-acc5-cbdec72e7075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:51.895327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56885b-1c59-490b-acc5-cbdec72e7075 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:51.895607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56885b-1c59-490b-acc5-cbdec72e7075 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:51.895874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56885b-1c59-490b-acc5-cbdec72e7075 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:51.896715 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56885b-1c59-490b-acc5-cbdec72e7075 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:51.897021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56885b-1c59-490b-acc5-cbdec72e7075 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:51.898471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56885b-1c59-490b-acc5-cbdec72e7075 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:51.927813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 355/703] 10.222.0.79 () {56 vars in 1133 bytes} [Fri May 13 13:41:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:51.935078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23894a8-13ce-47e7-8239-b457b5235cdb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:52.172655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23894a8-13ce-47e7-8239-b457b5235cdb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:52.225575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be4598-927e-4d78-8bfb-01ccebaa883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:52.226305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be4598-927e-4d78-8bfb-01ccebaa883d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:52.226594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be4598-927e-4d78-8bfb-01ccebaa883d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:52.226911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be4598-927e-4d78-8bfb-01ccebaa883d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:52.335206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be4598-927e-4d78-8bfb-01ccebaa883d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:52.336694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be4598-927e-4d78-8bfb-01ccebaa883d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:52.493224 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 349/704] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:52.500492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56885b-1c59-490b-acc5-cbdec72e7075 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:52.637930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56885b-1c59-490b-acc5-cbdec72e7075 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:52.684636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de497aa9-0e5f-4f5d-971e-20295bd92d92 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:52.685345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de497aa9-0e5f-4f5d-971e-20295bd92d92 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:52.685822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de497aa9-0e5f-4f5d-971e-20295bd92d92 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:52.687723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 350/705] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:52.693146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cb6b6-d7fa-4aea-bac7-2700f36bc806 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:52.693683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cb6b6-d7fa-4aea-bac7-2700f36bc806 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:52.694255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cb6b6-d7fa-4aea-bac7-2700f36bc806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:52.694925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4c3fd-1331-4d93-b063-d7db3f980c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:52.695765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4c3fd-1331-4d93-b063-d7db3f980c01 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:52.696048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4c3fd-1331-4d93-b063-d7db3f980c01 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:52.696322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4c3fd-1331-4d93-b063-d7db3f980c01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:52.708497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4c3fd-1331-4d93-b063-d7db3f980c01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:52.709983 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4c3fd-1331-4d93-b063-d7db3f980c01 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-55b4c3fd-1331-4d93-b063-d7db3f980c01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:52.733277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:52.734335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:52.735297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:52.735297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:52.735297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:52.735297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:52.735938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 356/706] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:52.748058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4c3fd-1331-4d93-b063-d7db3f980c01 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:52.748966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-972cb6b6-d7fa-4aea-bac7-2700f36bc806 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:52.751210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-972cb6b6-d7fa-4aea-bac7-2700f36bc806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:52.862999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4c3fd-1331-4d93-b063-d7db3f980c01 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:52.904887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da56be8-dad6-4524-8fbd-d0677d2ba5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:52.905659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da56be8-dad6-4524-8fbd-d0677d2ba5e6 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:52.906054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da56be8-dad6-4524-8fbd-d0677d2ba5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:52.906345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da56be8-dad6-4524-8fbd-d0677d2ba5e6 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:52.907391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da56be8-dad6-4524-8fbd-d0677d2ba5e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:52.907705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da56be8-dad6-4524-8fbd-d0677d2ba5e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:52.909132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da56be8-dad6-4524-8fbd-d0677d2ba5e6 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:52.924852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 357/707] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:52.935229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da56be8-dad6-4524-8fbd-d0677d2ba5e6 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:52.949992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 351/708] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:52.976849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cb6b6-d7fa-4aea-bac7-2700f36bc806 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:53.069185 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da56be8-dad6-4524-8fbd-d0677d2ba5e6 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:53.110760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d5d57-0edb-4f25-aeab-3f805c65cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:53.111676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d5d57-0edb-4f25-aeab-3f805c65cf38 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:53.111957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d5d57-0edb-4f25-aeab-3f805c65cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:53.112255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d5d57-0edb-4f25-aeab-3f805c65cf38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:53.123924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d5d57-0edb-4f25-aeab-3f805c65cf38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:53.125572 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d5d57-0edb-4f25-aeab-3f805c65cf38 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-6b0d5d57-0edb-4f25-aeab-3f805c65cf38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:53.141777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:53.143802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:53.145592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:53.145592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:53.145592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:53.145592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:53.145592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:53.145592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:53.145592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:53.145592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:53.145592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 358/709] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:53.163610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d5d57-0edb-4f25-aeab-3f805c65cf38 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:53.267899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cb6b6-d7fa-4aea-bac7-2700f36bc806 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:53.336264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d5d57-0edb-4f25-aeab-3f805c65cf38 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:53.341937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a9a10-dd8f-4852-9c22-7fb4631bfcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:53.343261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a9a10-dd8f-4852-9c22-7fb4631bfcd2 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:53.343677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a9a10-dd8f-4852-9c22-7fb4631bfcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:53.343960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a9a10-dd8f-4852-9c22-7fb4631bfcd2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:53.344915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a9a10-dd8f-4852-9c22-7fb4631bfcd2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:53.347412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a9a10-dd8f-4852-9c22-7fb4631bfcd2 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:53.377617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d27a82-6d3c-40a8-b20a-2e947c31bc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:53.378304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d27a82-6d3c-40a8-b20a-2e947c31bc91 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:53.378588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d27a82-6d3c-40a8-b20a-2e947c31bc91 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:53.378904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d27a82-6d3c-40a8-b20a-2e947c31bc91 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:53.379796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d27a82-6d3c-40a8-b20a-2e947c31bc91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:53.380106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d27a82-6d3c-40a8-b20a-2e947c31bc91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:53.381589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d27a82-6d3c-40a8-b20a-2e947c31bc91 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-4b3a9a10-dd8f-4852-9c22-7fb4631bfcd2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:53.382179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:53.385264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:53.388270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 352/710] 10.222.0.79 () {56 vars in 1129 bytes} [Fri May 13 13:41:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:53.399175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a9a10-dd8f-4852-9c22-7fb4631bfcd2 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:53.401499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 359/711] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:53.408243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d27a82-6d3c-40a8-b20a-2e947c31bc91 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:53.590322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d27a82-6d3c-40a8-b20a-2e947c31bc91 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:53.620465 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a9a10-dd8f-4852-9c22-7fb4631bfcd2 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:53.715213 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c580752-36c5-4eb9-9838-c0e2eaac22b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:53.716526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c580752-36c5-4eb9-9838-c0e2eaac22b0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:53.716732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c580752-36c5-4eb9-9838-c0e2eaac22b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:53.717632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c580752-36c5-4eb9-9838-c0e2eaac22b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:53.719251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c580752-36c5-4eb9-9838-c0e2eaac22b0 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=24ad0dd9d265425087be2be8d245b7fc) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:53.719251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c580752-36c5-4eb9-9838-c0e2eaac22b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:53.722104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c580752-36c5-4eb9-9838-c0e2eaac22b0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:53.726795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039117e3-ef11-4f5c-a8b6-92996dc138e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:53.728365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039117e3-ef11-4f5c-a8b6-92996dc138e6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:53.729143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039117e3-ef11-4f5c-a8b6-92996dc138e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:53.729842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039117e3-ef11-4f5c-a8b6-92996dc138e6 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:53.730865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039117e3-ef11-4f5c-a8b6-92996dc138e6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:53.731348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039117e3-ef11-4f5c-a8b6-92996dc138e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:53.733089 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039117e3-ef11-4f5c-a8b6-92996dc138e6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:53.733484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 360/712] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:53 2022] GET /identity/v3/role_assignments?user.id=24ad0dd9d265425087be2be8d245b7fc&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 259 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:53.735831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fb494-3ff4-496e-9238-283d93a1dabd None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:53.736281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fb494-3ff4-496e-9238-283d93a1dabd None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:53.736855 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fb494-3ff4-496e-9238-283d93a1dabd None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:53.745929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 361/713] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:53.746800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 353/714] 10.222.0.79 () {56 vars in 1124 bytes} [Fri May 13 13:41:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:53.753237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27517b40-909a-45b0-b474-3d3a0314f68b None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:53.753681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27517b40-909a-45b0-b474-3d3a0314f68b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:53.754190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27517b40-909a-45b0-b474-3d3a0314f68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:53.764297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039117e3-ef11-4f5c-a8b6-92996dc138e6 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:53.804569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-27517b40-909a-45b0-b474-3d3a0314f68b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:53.807369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-27517b40-909a-45b0-b474-3d3a0314f68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:53.956809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 362/715] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:54.014476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27517b40-909a-45b0-b474-3d3a0314f68b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:54.020671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039117e3-ef11-4f5c-a8b6-92996dc138e6 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:54.092981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70134bc3-aacf-43e4-9beb-391d77c269b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:54.094382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70134bc3-aacf-43e4-9beb-391d77c269b5 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:54.094695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70134bc3-aacf-43e4-9beb-391d77c269b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:54.095005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70134bc3-aacf-43e4-9beb-391d77c269b5 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:54.096289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70134bc3-aacf-43e4-9beb-391d77c269b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:54.096664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70134bc3-aacf-43e4-9beb-391d77c269b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:54.099126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70134bc3-aacf-43e4-9beb-391d77c269b5 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:54.107695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 354/716] 10.222.0.79 () {56 vars in 1124 bytes} [Fri May 13 13:41:53 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:54.198938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92eea3-b85c-41d6-9928-a67fc63455d0 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:54.199432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92eea3-b85c-41d6-9928-a67fc63455d0 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:54.199935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92eea3-b85c-41d6-9928-a67fc63455d0 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:54.202875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 355/717] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:41:54.212316 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7420cd-fe1f-4596-a7bd-ad70d70bd277 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:54.212699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7420cd-fe1f-4596-a7bd-ad70d70bd277 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:54.213170 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7420cd-fe1f-4596-a7bd-ad70d70bd277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:54.213647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27517b40-909a-45b0-b474-3d3a0314f68b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:54.239529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-9e7420cd-fe1f-4596-a7bd-ad70d70bd277 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:54.241620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9e7420cd-fe1f-4596-a7bd-ad70d70bd277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:54.264624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f44804-ebc2-47ca-b6f9-c13bd3157d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:54.265611 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f44804-ebc2-47ca-b6f9-c13bd3157d8d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:54.265962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f44804-ebc2-47ca-b6f9-c13bd3157d8d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:54.266210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f44804-ebc2-47ca-b6f9-c13bd3157d8d None admin] PATH_INFO: `/v3/services/network` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:54.266927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f44804-ebc2-47ca-b6f9-c13bd3157d8d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:54.268832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f44804-ebc2-47ca-b6f9-c13bd3157d8d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-89f44804-ebc2-47ca-b6f9-c13bd3157d8d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:54.293273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:54.297146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:54.305164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 363/718] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:54 2022] GET /identity/v3/services/network => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:54.308416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f44804-ebc2-47ca-b6f9-c13bd3157d8d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:54.375405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 356/719] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:54.460615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7420cd-fe1f-4596-a7bd-ad70d70bd277 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:54.557424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f44804-ebc2-47ca-b6f9-c13bd3157d8d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:54.612687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe1948-ef1a-4959-a132-d02fa58e8343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:54.616719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe1948-ef1a-4959-a132-d02fa58e8343 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:54.617192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe1948-ef1a-4959-a132-d02fa58e8343 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:54.617633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe1948-ef1a-4959-a132-d02fa58e8343 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:54.618902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe1948-ef1a-4959-a132-d02fa58e8343 None admin] RBAC: Adding query filter params (name=network) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:54.619408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe1948-ef1a-4959-a132-d02fa58e8343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:54.621811 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe1948-ef1a-4959-a132-d02fa58e8343 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:54.632953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 364/720] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:41:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:54.655995 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe1948-ef1a-4959-a132-d02fa58e8343 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:54.668657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7420cd-fe1f-4596-a7bd-ad70d70bd277 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:54.742955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef588ed-436b-4c09-b845-2cc965214b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:54.744408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef588ed-436b-4c09-b845-2cc965214b88 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:54.744902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef588ed-436b-4c09-b845-2cc965214b88 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:54.745381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef588ed-436b-4c09-b845-2cc965214b88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:54.779490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef588ed-436b-4c09-b845-2cc965214b88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-4ef588ed-436b-4c09-b845-2cc965214b88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:54.787988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:54.791679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:54.794556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 357/721] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:41:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:54.803875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db815bb-eae4-4fca-aad7-5b2600e14f85 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:54.803875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db815bb-eae4-4fca-aad7-5b2600e14f85 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:54.803875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db815bb-eae4-4fca-aad7-5b2600e14f85 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:54.805428 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 358/722] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:54.807142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db815bb-eae4-4fca-aad7-5b2600e14f85 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:54.846777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe1948-ef1a-4959-a132-d02fa58e8343 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:54.886265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8e734-411f-4fd3-aab5-e07a771a83d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:54.886977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8e734-411f-4fd3-aab5-e07a771a83d3 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:54.887276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8e734-411f-4fd3-aab5-e07a771a83d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:54.887520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8e734-411f-4fd3-aab5-e07a771a83d3 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:54.888242 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8e734-411f-4fd3-aab5-e07a771a83d3 None admin] RBAC: Adding query filter params (type=network) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:54.888544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8e734-411f-4fd3-aab5-e07a771a83d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:54.890039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8e734-411f-4fd3-aab5-e07a771a83d3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:54.897313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 365/723] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:41:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:54.898777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7e3b-d7ef-44a9-87a7-99e54efdaabd None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:54.899146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7e3b-d7ef-44a9-87a7-99e54efdaabd None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:54.899429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7e3b-d7ef-44a9-87a7-99e54efdaabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:54.943655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e8de7e3b-d7ef-44a9-87a7-99e54efdaabd None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:54.947552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e8de7e3b-d7ef-44a9-87a7-99e54efdaabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:55.100643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db815bb-eae4-4fca-aad7-5b2600e14f85 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:55.108235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 366/724] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:55.142702 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de7e3b-d7ef-44a9-87a7-99e54efdaabd None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:55.182729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f938ac-ba67-449b-87a1-8aebe277c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:55.183422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f938ac-ba67-449b-87a1-8aebe277c66a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:55.183705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f938ac-ba67-449b-87a1-8aebe277c66a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:55.183975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f938ac-ba67-449b-87a1-8aebe277c66a None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:55.184764 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f938ac-ba67-449b-87a1-8aebe277c66a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:55.185110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f938ac-ba67-449b-87a1-8aebe277c66a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:55.186534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f938ac-ba67-449b-87a1-8aebe277c66a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:55.204655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 359/725] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:55.211348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f938ac-ba67-449b-87a1-8aebe277c66a None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:55.386204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de7e3b-d7ef-44a9-87a7-99e54efdaabd None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:55.423694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f938ac-ba67-449b-87a1-8aebe277c66a None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:55.486043 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adfc3d-23b9-4d44-9fb2-20da52842c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:55.487449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adfc3d-23b9-4d44-9fb2-20da52842c22 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:55.487827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adfc3d-23b9-4d44-9fb2-20da52842c22 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:55.488189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adfc3d-23b9-4d44-9fb2-20da52842c22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:55.526184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adfc3d-23b9-4d44-9fb2-20da52842c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:55.537603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521bb46-aa7d-485e-8f1c-1b97f902b305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:55.538464 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521bb46-aa7d-485e-8f1c-1b97f902b305 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:55.538818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521bb46-aa7d-485e-8f1c-1b97f902b305 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:55.539140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521bb46-aa7d-485e-8f1c-1b97f902b305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-94adfc3d-23b9-4d44-9fb2-20da52842c22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:55.548946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:55.551430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:55.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 367/726] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:41:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:55.564448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521bb46-aa7d-485e-8f1c-1b97f902b305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:55.564448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521bb46-aa7d-485e-8f1c-1b97f902b305 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:55.568762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adfc3d-23b9-4d44-9fb2-20da52842c22 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-0521bb46-aa7d-485e-8f1c-1b97f902b305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:55.604961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:55.606770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:55.608375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:55.608375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:55.608375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:55.608375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:55.612369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 360/727] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:55.632694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521bb46-aa7d-485e-8f1c-1b97f902b305 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:55.717596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adfc3d-23b9-4d44-9fb2-20da52842c22 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:55.772590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56700e2-e9d0-4693-80a0-4896e49d45ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:55.775616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56700e2-e9d0-4693-80a0-4896e49d45ef None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:55.776517 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56700e2-e9d0-4693-80a0-4896e49d45ef None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:55.777513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56700e2-e9d0-4693-80a0-4896e49d45ef None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:55.781868 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56700e2-e9d0-4693-80a0-4896e49d45ef None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:55.782797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56700e2-e9d0-4693-80a0-4896e49d45ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:55.788336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56700e2-e9d0-4693-80a0-4896e49d45ef None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:55.815168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 368/728] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:41:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:55.824408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56700e2-e9d0-4693-80a0-4896e49d45ef None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:55.883104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521bb46-aa7d-485e-8f1c-1b97f902b305 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:55.924622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966d04-49d4-479c-bc6a-fd08384117ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:55.926443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966d04-49d4-479c-bc6a-fd08384117ff None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:55.929413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966d04-49d4-479c-bc6a-fd08384117ff None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:55.931405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966d04-49d4-479c-bc6a-fd08384117ff None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:55.932978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed966d04-49d4-479c-bc6a-fd08384117ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:55.934912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed966d04-49d4-479c-bc6a-fd08384117ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:55.937903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed966d04-49d4-479c-bc6a-fd08384117ff None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:55.987902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 361/729] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:55.996546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966d04-49d4-479c-bc6a-fd08384117ff None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:56.151484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966d04-49d4-479c-bc6a-fd08384117ff None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:56.270852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7e4dd-a07d-4912-a86e-88b66d00fa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:56.270852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7e4dd-a07d-4912-a86e-88b66d00fa8f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:56.270852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7e4dd-a07d-4912-a86e-88b66d00fa8f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:56.270852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7e4dd-a07d-4912-a86e-88b66d00fa8f None admin] PATH_INFO: `/v3/users/swift` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:56.276175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56700e2-e9d0-4693-80a0-4896e49d45ef None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:56.289670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7e4dd-a07d-4912-a86e-88b66d00fa8f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:56.292710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7e4dd-a07d-4912-a86e-88b66d00fa8f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-1eb7e4dd-a07d-4912-a86e-88b66d00fa8f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:56.350663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:56.352453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:56.354281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:56.354281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:56.354281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:56.354281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:56.354281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:56.354281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:56.354281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:56.354281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:56.354281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:41:56.354281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:56.360925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 362/730] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:41:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:56.366772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf5ff3-3e8b-46a2-ba32-303651d93bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:56.371011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf5ff3-3e8b-46a2-ba32-303651d93bd5 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:56.371617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf5ff3-3e8b-46a2-ba32-303651d93bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:56.372217 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf5ff3-3e8b-46a2-ba32-303651d93bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:56.383815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7e4dd-a07d-4912-a86e-88b66d00fa8f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:56.397706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf5ff3-3e8b-46a2-ba32-303651d93bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:56.400810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf5ff3-3e8b-46a2-ba32-303651d93bd5 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-34cf5ff3-3e8b-46a2-ba32-303651d93bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:56.447281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:56.449130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:56.451134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:56.451134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:56.451134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:56.451134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:56.456575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 369/731] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 633 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:56.469005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf5ff3-3e8b-46a2-ba32-303651d93bd5 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:56.636665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf5ff3-3e8b-46a2-ba32-303651d93bd5 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:56.701294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6486fe-4bec-4041-9a5a-bd23d082f754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:56.702450 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6486fe-4bec-4041-9a5a-bd23d082f754 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:56.703003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6486fe-4bec-4041-9a5a-bd23d082f754 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:56.703511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6486fe-4bec-4041-9a5a-bd23d082f754 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:56.710846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6486fe-4bec-4041-9a5a-bd23d082f754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:56.710846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6486fe-4bec-4041-9a5a-bd23d082f754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:56.710846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6486fe-4bec-4041-9a5a-bd23d082f754 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:56.731461 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 370/732] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:56.738122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7e4dd-a07d-4912-a86e-88b66d00fa8f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:56.739890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6486fe-4bec-4041-9a5a-bd23d082f754 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:56.803016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179447-b03d-43b4-a162-6d7f986bbfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:56.804723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179447-b03d-43b4-a162-6d7f986bbfff None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:56.808090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179447-b03d-43b4-a162-6d7f986bbfff None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:56.808991 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179447-b03d-43b4-a162-6d7f986bbfff None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:56.811113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1179447-b03d-43b4-a162-6d7f986bbfff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:56.815157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1179447-b03d-43b4-a162-6d7f986bbfff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:56.818935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1179447-b03d-43b4-a162-6d7f986bbfff None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:56.901635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 363/733] 10.222.0.79 () {56 vars in 1135 bytes} [Fri May 13 13:41:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 518 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:56.911882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179447-b03d-43b4-a162-6d7f986bbfff None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:56.963732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6486fe-4bec-4041-9a5a-bd23d082f754 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:57.026090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c0293-25ec-4b72-80af-5f117632e43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:57.027906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c0293-25ec-4b72-80af-5f117632e43d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:57.028571 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c0293-25ec-4b72-80af-5f117632e43d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:57.029099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c0293-25ec-4b72-80af-5f117632e43d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:57.042986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c0293-25ec-4b72-80af-5f117632e43d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:57.045762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c0293-25ec-4b72-80af-5f117632e43d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-122c0293-25ec-4b72-80af-5f117632e43d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:41:57.076788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:57.077832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:57.078802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:57.078802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:57.078802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:41:57.078802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:41:57.078802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:41:57.078802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:41:57.078802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:41:57.078802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:41:57.078802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:41:57.078802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:57.082099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 371/734] 10.222.0.79 () {56 vars in 1117 bytes} [Fri May 13 13:41:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:57.089319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c0293-25ec-4b72-80af-5f117632e43d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:57.156450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179447-b03d-43b4-a162-6d7f986bbfff None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:57.203068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29570189-5a5b-461e-b515-3f376a01aeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:57.203752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29570189-5a5b-461e-b515-3f376a01aeac None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:57.204077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29570189-5a5b-461e-b515-3f376a01aeac None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:57.204438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29570189-5a5b-461e-b515-3f376a01aeac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:57.374816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29570189-5a5b-461e-b515-3f376a01aeac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:57.376883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29570189-5a5b-461e-b515-3f376a01aeac None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:57.415478 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c0293-25ec-4b72-80af-5f117632e43d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:57.557383 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 364/735] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:41:57.563815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29570189-5a5b-461e-b515-3f376a01aeac None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:57.696329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d86e7a-8ed3-4d0c-ab33-ac7390c3fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:57.698256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d86e7a-8ed3-4d0c-ab33-ac7390c3fa82 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:57.699297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d86e7a-8ed3-4d0c-ab33-ac7390c3fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:57.700074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d86e7a-8ed3-4d0c-ab33-ac7390c3fa82 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:57.702789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d86e7a-8ed3-4d0c-ab33-ac7390c3fa82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:57.703614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d86e7a-8ed3-4d0c-ab33-ac7390c3fa82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:57.707628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d86e7a-8ed3-4d0c-ab33-ac7390c3fa82 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:57.745410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 372/736] 10.222.0.79 () {56 vars in 1149 bytes} [Fri May 13 13:41:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 657 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:57.751419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d86e7a-8ed3-4d0c-ab33-ac7390c3fa82 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:57.797678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29570189-5a5b-461e-b515-3f376a01aeac None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:57.850853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc906eac-93dc-494a-9c0c-e500fdc9fc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:57.851719 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc906eac-93dc-494a-9c0c-e500fdc9fc83 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:57.852105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc906eac-93dc-494a-9c0c-e500fdc9fc83 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:57.852375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc906eac-93dc-494a-9c0c-e500fdc9fc83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:57.881503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc906eac-93dc-494a-9c0c-e500fdc9fc83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:57.883450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc906eac-93dc-494a-9c0c-e500fdc9fc83 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-dc906eac-93dc-494a-9c0c-e500fdc9fc83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:57.906127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:57.907948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:57.909726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:57.909726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:57.909726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:57.909726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:57.909726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 365/737] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:41:57.930800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc906eac-93dc-494a-9c0c-e500fdc9fc83 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:58.044574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d86e7a-8ed3-4d0c-ab33-ac7390c3fa82 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:58.121834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69418e60-a060-4543-89c6-f77e68a2974b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:58.123027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69418e60-a060-4543-89c6-f77e68a2974b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:58.123492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69418e60-a060-4543-89c6-f77e68a2974b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:58.123973 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69418e60-a060-4543-89c6-f77e68a2974b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:58.151055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc906eac-93dc-494a-9c0c-e500fdc9fc83 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:58.210180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c1b6c-5c4d-4903-8392-25fb4ab31f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:58.211288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c1b6c-5c4d-4903-8392-25fb4ab31f24 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:58.211638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c1b6c-5c4d-4903-8392-25fb4ab31f24 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:58.212113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c1b6c-5c4d-4903-8392-25fb4ab31f24 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:58.213526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c1b6c-5c4d-4903-8392-25fb4ab31f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:58.214048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c1b6c-5c4d-4903-8392-25fb4ab31f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:58.216128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c1b6c-5c4d-4903-8392-25fb4ab31f24 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:58.242870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 366/738] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:41:58.242870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0aa9be-a770-46ec-948c-5bb6b2b2f0e5 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:58.242870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0aa9be-a770-46ec-948c-5bb6b2b2f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:58.242870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0aa9be-a770-46ec-948c-5bb6b2b2f0e5 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:58.246705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 367/739] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:58.247320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54431b3b-db14-4adb-9af2-23f30f5629c1 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:58.247833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54431b3b-db14-4adb-9af2-23f30f5629c1 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:58.248398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54431b3b-db14-4adb-9af2-23f30f5629c1 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:58.251545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 368/740] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:58.251928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69418e60-a060-4543-89c6-f77e68a2974b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:58.255015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54431b3b-db14-4adb-9af2-23f30f5629c1 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:58.255192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69418e60-a060-4543-89c6-f77e68a2974b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:58.400901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54431b3b-db14-4adb-9af2-23f30f5629c1 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:58.437860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 373/741] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:41:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:41:58.437860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eaa4e-99f6-4166-b276-51225e81245f None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:58.437860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eaa4e-99f6-4166-b276-51225e81245f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:58.437860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eaa4e-99f6-4166-b276-51225e81245f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:58.463276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e93eaa4e-99f6-4166-b276-51225e81245f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:58.465327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e93eaa4e-99f6-4166-b276-51225e81245f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:58.471245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96285f87-49cf-46c5-9e0c-80e657b5d18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:58.472020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96285f87-49cf-46c5-9e0c-80e657b5d18e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:58.472581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96285f87-49cf-46c5-9e0c-80e657b5d18e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:58.473473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96285f87-49cf-46c5-9e0c-80e657b5d18e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:58.486285 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96285f87-49cf-46c5-9e0c-80e657b5d18e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:58.489643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96285f87-49cf-46c5-9e0c-80e657b5d18e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-96285f87-49cf-46c5-9e0c-80e657b5d18e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:58.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:58.514177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 369/742] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe82f-dbbc-48c7-86e8-ba36193cd360 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:58.522175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe82f-dbbc-48c7-86e8-ba36193cd360 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:58.527253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe82f-dbbc-48c7-86e8-ba36193cd360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:58.546158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7ddfe82f-dbbc-48c7-86e8-ba36193cd360 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:58.547480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7ddfe82f-dbbc-48c7-86e8-ba36193cd360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:58.594747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 374/743] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:58.594747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93eaa4e-99f6-4166-b276-51225e81245f None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:58.659027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 370/744] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:58.661492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe82f-dbbc-48c7-86e8-ba36193cd360 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:58.762381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93eaa4e-99f6-4166-b276-51225e81245f None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:58.810668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6d2e1-c0b4-44a3-bb22-039f6ecb12f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:58.811414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d2e1-c0b4-44a3-bb22-039f6ecb12f8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:58.811693 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d2e1-c0b4-44a3-bb22-039f6ecb12f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:58.811968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d2e1-c0b4-44a3-bb22-039f6ecb12f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:58.824676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6d2e1-c0b4-44a3-bb22-039f6ecb12f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:58.827822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6d2e1-c0b4-44a3-bb22-039f6ecb12f8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:58.845372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe82f-dbbc-48c7-86e8-ba36193cd360 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-eaa6d2e1-c0b4-44a3-bb22-039f6ecb12f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:41:58.856437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:58.858229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:58.860025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:41:58.860025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:41:58.860025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:41:58.860025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:58.860025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 375/745] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:58.860025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6d2e1-c0b4-44a3-bb22-039f6ecb12f8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:58.904351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b92c9-5784-444f-9d7f-55eec1ac592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:58.905297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b92c9-5784-444f-9d7f-55eec1ac592b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:58.905698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b92c9-5784-444f-9d7f-55eec1ac592b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:58.906095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b92c9-5784-444f-9d7f-55eec1ac592b None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:58.907432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b92c9-5784-444f-9d7f-55eec1ac592b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:58.907908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b92c9-5784-444f-9d7f-55eec1ac592b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:58.910737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b92c9-5784-444f-9d7f-55eec1ac592b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:58.931393 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 371/746] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:58.932759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b92c9-5784-444f-9d7f-55eec1ac592b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:58.987029 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6d2e1-c0b4-44a3-bb22-039f6ecb12f8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:59.038351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22a088-3779-4fdf-8efb-018a76770620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:59.039287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22a088-3779-4fdf-8efb-018a76770620 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:59.039662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22a088-3779-4fdf-8efb-018a76770620 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:59.040025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22a088-3779-4fdf-8efb-018a76770620 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:59.040886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22a088-3779-4fdf-8efb-018a76770620 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:59.042389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22a088-3779-4fdf-8efb-018a76770620 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:59.061477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 376/747] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:41:58 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:41:59.063131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c2b28-4189-4f1c-83f8-4ecea08e88ae None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:59.063546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c2b28-4189-4f1c-83f8-4ecea08e88ae None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:59.063910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c2b28-4189-4f1c-83f8-4ecea08e88ae None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:59.065453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 377/748] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:41:59.066966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c2b28-4189-4f1c-83f8-4ecea08e88ae None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:59.112830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b92c9-5784-444f-9d7f-55eec1ac592b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:59.182540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1411902-8595-429d-a71e-04a5ba6bf004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:59.183461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1411902-8595-429d-a71e-04a5ba6bf004 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:59.183814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1411902-8595-429d-a71e-04a5ba6bf004 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:59.183997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1411902-8595-429d-a71e-04a5ba6bf004 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:59.196890 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1411902-8595-429d-a71e-04a5ba6bf004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-f1411902-8595-429d-a71e-04a5ba6bf004 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:41:59.205666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:59.207627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:41:59.209007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 372/749] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:59.210431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1411902-8595-429d-a71e-04a5ba6bf004 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:59.255916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c2b28-4189-4f1c-83f8-4ecea08e88ae None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:59.316821 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa88a6d-544d-4917-b2cd-2dffaf7fc211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:59.318211 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88a6d-544d-4917-b2cd-2dffaf7fc211 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:59.318923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88a6d-544d-4917-b2cd-2dffaf7fc211 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:59.319275 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88a6d-544d-4917-b2cd-2dffaf7fc211 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:59.320477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa88a6d-544d-4917-b2cd-2dffaf7fc211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:59.320995 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa88a6d-544d-4917-b2cd-2dffaf7fc211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:59.323813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa88a6d-544d-4917-b2cd-2dffaf7fc211 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:59.345231 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 378/750] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:59.347915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877d8d2-71b5-4543-bcd7-d89d06bed05c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:59.348473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877d8d2-71b5-4543-bcd7-d89d06bed05c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:59.349495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877d8d2-71b5-4543-bcd7-d89d06bed05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:59.368085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e877d8d2-71b5-4543-bcd7-d89d06bed05c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:41:59.369793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e877d8d2-71b5-4543-bcd7-d89d06bed05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:41:59.395763 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1411902-8595-429d-a71e-04a5ba6bf004 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:59.461838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593bf2a-cdf7-4695-ab70-3f86e1b61e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:59.462963 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bf2a-cdf7-4695-ab70-3f86e1b61e27 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:59.463437 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bf2a-cdf7-4695-ab70-3f86e1b61e27 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:59.463929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bf2a-cdf7-4695-ab70-3f86e1b61e27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:59.465569 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593bf2a-cdf7-4695-ab70-3f86e1b61e27 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=4139cfe906bd4c7d94226ab589577627) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:59.467038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593bf2a-cdf7-4695-ab70-3f86e1b61e27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:59.469025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593bf2a-cdf7-4695-ab70-3f86e1b61e27 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:59.490754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 373/751] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:41:59 2022] GET /identity/v3/role_assignments?user.id=4139cfe906bd4c7d94226ab589577627&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:59.490754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593bf2a-cdf7-4695-ab70-3f86e1b61e27 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:59.498410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 379/752] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:41:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:41:59.499383 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877d8d2-71b5-4543-bcd7-d89d06bed05c None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:59.671356 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877d8d2-71b5-4543-bcd7-d89d06bed05c None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:59.702128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593bf2a-cdf7-4695-ab70-3f86e1b61e27 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:59.729851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3c0d6-582e-4738-bfa1-210fccf5f5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:59.731432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3c0d6-582e-4738-bfa1-210fccf5f5d1 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:59.731870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3c0d6-582e-4738-bfa1-210fccf5f5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:59.732266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3c0d6-582e-4738-bfa1-210fccf5f5d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:59.745132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3c0d6-582e-4738-bfa1-210fccf5f5d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:59.747808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3c0d6-582e-4738-bfa1-210fccf5f5d1 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:59.768857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cc0ce-1a26-48b9-b156-a9f89229c5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:59.770077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cc0ce-1a26-48b9-b156-a9f89229c5db None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:59.770578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cc0ce-1a26-48b9-b156-a9f89229c5db None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:59.771111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cc0ce-1a26-48b9-b156-a9f89229c5db None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:59.772589 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cc0ce-1a26-48b9-b156-a9f89229c5db None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:41:59.773126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cc0ce-1a26-48b9-b156-a9f89229c5db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-89a3c0d6-582e-4738-bfa1-210fccf5f5d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:41:59.773773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:41:59.777066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:41:59.780468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:41:59.780468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:41:59.780468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:41:59.780468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:41:59.780468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:41:59.780468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:41:59.780468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:41:59.780468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:41:59.787015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cc0ce-1a26-48b9-b156-a9f89229c5db None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:59.787134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 380/753] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:41:59.787134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3c0d6-582e-4738-bfa1-210fccf5f5d1 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:59.798968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 374/754] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:41:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:41:59.801581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cc0ce-1a26-48b9-b156-a9f89229c5db None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:59.913445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3c0d6-582e-4738-bfa1-210fccf5f5d1 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:41:59.954188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca815bfc-8a7c-4df9-9123-6b94e1eaec90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:41:59.955223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815bfc-8a7c-4df9-9123-6b94e1eaec90 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:41:59.955659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815bfc-8a7c-4df9-9123-6b94e1eaec90 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:41:59.956087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815bfc-8a7c-4df9-9123-6b94e1eaec90 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:41:59.957028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca815bfc-8a7c-4df9-9123-6b94e1eaec90 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:41:59.958615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca815bfc-8a7c-4df9-9123-6b94e1eaec90 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:41:59.984181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 381/755] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:41:59 2022] POST /identity/v3/services => generated 240 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:41:59.986307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca815bfc-8a7c-4df9-9123-6b94e1eaec90 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:41:59.995641 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cc0ce-1a26-48b9-b156-a9f89229c5db None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:00.073198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8e6d8-d197-4733-8226-9a38a0e8d90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:00.074438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8e6d8-d197-4733-8226-9a38a0e8d90a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:00.074969 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8e6d8-d197-4733-8226-9a38a0e8d90a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:00.076159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8e6d8-d197-4733-8226-9a38a0e8d90a None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:00.077317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8e6d8-d197-4733-8226-9a38a0e8d90a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:00.077985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8e6d8-d197-4733-8226-9a38a0e8d90a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:00.080713 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8e6d8-d197-4733-8226-9a38a0e8d90a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:00.110759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 375/756] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:41:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:00.118452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8e6d8-d197-4733-8226-9a38a0e8d90a None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:00.176380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca815bfc-8a7c-4df9-9123-6b94e1eaec90 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:00.232243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73179c0-efc8-4f9a-a8da-b1570ecf3a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:00.233447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73179c0-efc8-4f9a-a8da-b1570ecf3a05 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:00.233952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73179c0-efc8-4f9a-a8da-b1570ecf3a05 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:00.234438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73179c0-efc8-4f9a-a8da-b1570ecf3a05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:00.247729 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73179c0-efc8-4f9a-a8da-b1570ecf3a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:00.249640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73179c0-efc8-4f9a-a8da-b1570ecf3a05 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-c73179c0-efc8-4f9a-a8da-b1570ecf3a05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:00.270770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:00.271772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:00.272682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:00.272682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:00.272682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:00.272682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:00.275227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 382/757] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:00.297990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73179c0-efc8-4f9a-a8da-b1570ecf3a05 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:00.323049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8e6d8-d197-4733-8226-9a38a0e8d90a None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:00.389822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d3f23-cc82-4f8a-a0e8-f7faf5a46919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:00.390966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d3f23-cc82-4f8a-a0e8-f7faf5a46919 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:00.391434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d3f23-cc82-4f8a-a0e8-f7faf5a46919 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:00.391888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d3f23-cc82-4f8a-a0e8-f7faf5a46919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:00.393783 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d3f23-cc82-4f8a-a0e8-f7faf5a46919 None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=598c323e8ed5455cbe4a568b360d8eab) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:00.393919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d3f23-cc82-4f8a-a0e8-f7faf5a46919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:00.397104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d3f23-cc82-4f8a-a0e8-f7faf5a46919 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:00.410930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 376/758] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:42:00 2022] GET /identity/v3/role_assignments?user.id=598c323e8ed5455cbe4a568b360d8eab&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:00.490615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73179c0-efc8-4f9a-a8da-b1570ecf3a05 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:00.566895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e077e63-3c25-4fac-9acc-6329aea683a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:00.568131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077e63-3c25-4fac-9acc-6329aea683a0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:00.568628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077e63-3c25-4fac-9acc-6329aea683a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:00.569163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077e63-3c25-4fac-9acc-6329aea683a0 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:00.570582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e077e63-3c25-4fac-9acc-6329aea683a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:00.571124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e077e63-3c25-4fac-9acc-6329aea683a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:00.573839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e077e63-3c25-4fac-9acc-6329aea683a0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:00.600264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 383/759] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:00.608610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d3f23-cc82-4f8a-a0e8-f7faf5a46919 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:00.819605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d3f23-cc82-4f8a-a0e8-f7faf5a46919 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:00.885502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68249c-77cb-4246-b976-77ad046a356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:00.886889 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68249c-77cb-4246-b976-77ad046a356b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:00.887366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68249c-77cb-4246-b976-77ad046a356b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:00.887836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68249c-77cb-4246-b976-77ad046a356b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:00.902206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68249c-77cb-4246-b976-77ad046a356b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:00.904889 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68249c-77cb-4246-b976-77ad046a356b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-dc68249c-77cb-4246-b976-77ad046a356b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:00.932299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:00.934268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:00.938260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:00.938260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:00.938260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:00.938260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:00.938260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:00.938260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:00.938260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:00.938260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:00.938260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:42:00.938260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:00.942812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 377/760] 10.222.0.79 () {56 vars in 1093 bytes} [Fri May 13 13:42:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:00.943047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e077e63-3c25-4fac-9acc-6329aea683a0 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:01.203681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e077e63-3c25-4fac-9acc-6329aea683a0 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:01.282455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5b25a-8532-4976-abd6-83be5b5ee646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:01.283952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5b25a-8532-4976-abd6-83be5b5ee646 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:01.284630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5b25a-8532-4976-abd6-83be5b5ee646 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:01.285859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5b25a-8532-4976-abd6-83be5b5ee646 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:01.287672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5b25a-8532-4976-abd6-83be5b5ee646 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:01.288392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5b25a-8532-4976-abd6-83be5b5ee646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:01.291044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5b25a-8532-4976-abd6-83be5b5ee646 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:01.317888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 384/761] 10.222.0.79 () {56 vars in 1133 bytes} [Fri May 13 13:42:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:01.327070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68249c-77cb-4246-b976-77ad046a356b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:01.548323 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68249c-77cb-4246-b976-77ad046a356b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:01.621339 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0767e97-4bc5-416a-90c9-4ad830d9c881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:01.623064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767e97-4bc5-416a-90c9-4ad830d9c881 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:01.623782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767e97-4bc5-416a-90c9-4ad830d9c881 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:01.624484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767e97-4bc5-416a-90c9-4ad830d9c881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:01.648356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0767e97-4bc5-416a-90c9-4ad830d9c881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:01.650258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0767e97-4bc5-416a-90c9-4ad830d9c881 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-d0767e97-4bc5-416a-90c9-4ad830d9c881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:01.679485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:01.680762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:01.682032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:01.682032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:01.682032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:01.682032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:01.692649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 378/762] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:01.701194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5b25a-8532-4976-abd6-83be5b5ee646 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:01.884720 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5b25a-8532-4976-abd6-83be5b5ee646 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:01.950306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db39bd-0b21-4ea6-81e5-96ec70cd7ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:01.951172 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db39bd-0b21-4ea6-81e5-96ec70cd7ee0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:01.951456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db39bd-0b21-4ea6-81e5-96ec70cd7ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:01.951731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db39bd-0b21-4ea6-81e5-96ec70cd7ee0 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:01.952588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db39bd-0b21-4ea6-81e5-96ec70cd7ee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:01.952895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db39bd-0b21-4ea6-81e5-96ec70cd7ee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:01.954336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db39bd-0b21-4ea6-81e5-96ec70cd7ee0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:01.979563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 385/763] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:01.985791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0767e97-4bc5-416a-90c9-4ad830d9c881 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:02.148081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0767e97-4bc5-416a-90c9-4ad830d9c881 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:02.198411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12b106-255e-497d-8689-2c95a76fcb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:02.199254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12b106-255e-497d-8689-2c95a76fcb9a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:02.199531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12b106-255e-497d-8689-2c95a76fcb9a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:02.199803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12b106-255e-497d-8689-2c95a76fcb9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:02.207942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12b106-255e-497d-8689-2c95a76fcb9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:02.210038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12b106-255e-497d-8689-2c95a76fcb9a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-6b12b106-255e-497d-8689-2c95a76fcb9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:02.234451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:02.236114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:02.238911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:02.238911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:02.238911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:02.238911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:02.238911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:02.238911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:02.238911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:02.238911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:02.238911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 379/764] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:02.247257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db39bd-0b21-4ea6-81e5-96ec70cd7ee0 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:02.402869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db39bd-0b21-4ea6-81e5-96ec70cd7ee0 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:02.440060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaad1f2-6e3e-4247-bcbc-858ad88c5cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:02.440733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaad1f2-6e3e-4247-bcbc-858ad88c5cb0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:02.441122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaad1f2-6e3e-4247-bcbc-858ad88c5cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:02.441408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaad1f2-6e3e-4247-bcbc-858ad88c5cb0 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:02.442264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaad1f2-6e3e-4247-bcbc-858ad88c5cb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:02.442573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaad1f2-6e3e-4247-bcbc-858ad88c5cb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:02.444101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaad1f2-6e3e-4247-bcbc-858ad88c5cb0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:02.462427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 386/765] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:02.470398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12b106-255e-497d-8689-2c95a76fcb9a None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:02.637671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12b106-255e-497d-8689-2c95a76fcb9a None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:02.685306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b62b00-b616-4988-b198-19dfdb38ff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:02.686227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b62b00-b616-4988-b198-19dfdb38ff80 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:02.686506 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b62b00-b616-4988-b198-19dfdb38ff80 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:02.686826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b62b00-b616-4988-b198-19dfdb38ff80 None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/24ad0dd9d265425087be2be8d245b7fc/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:02.689011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b62b00-b616-4988-b198-19dfdb38ff80 None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=24ad0dd9d265425087be2be8d245b7fc, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:02.690498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b62b00-b616-4988-b198-19dfdb38ff80 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:02.708914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 380/766] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:42:02 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/24ad0dd9d265425087be2be8d245b7fc/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:42:02.987364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395f397-66a7-483b-8d36-ab9eecc9df89 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:02.988814 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395f397-66a7-483b-8d36-ab9eecc9df89 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:02.988814 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395f397-66a7-483b-8d36-ab9eecc9df89 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:02.990202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 387/767] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:03.003260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd79bc-034b-4b71-8b66-a95d1b80b7f0 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:03.004213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd79bc-034b-4b71-8b66-a95d1b80b7f0 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:03.004448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd79bc-034b-4b71-8b66-a95d1b80b7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:03.028310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-76cd79bc-034b-4b71-8b66-a95d1b80b7f0 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:03.029535 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-76cd79bc-034b-4b71-8b66-a95d1b80b7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:03.125327 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 381/768] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:03.174781 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395f397-66a7-483b-8d36-ab9eecc9df89 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:03.262088 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453ec45-afb0-4b7c-b760-29a797617f2b None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:03.262486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453ec45-afb0-4b7c-b760-29a797617f2b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:03.262486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453ec45-afb0-4b7c-b760-29a797617f2b None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:03.264715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 382/769] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:03.290937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2905c-a31b-44cd-b121-5182f860de2f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:03.291470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2905c-a31b-44cd-b121-5182f860de2f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:03.291849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2905c-a31b-44cd-b121-5182f860de2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:03.313286 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-83f2905c-a31b-44cd-b121-5182f860de2f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:03.314623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-83f2905c-a31b-44cd-b121-5182f860de2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:03.393123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 383/770] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:03.429163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2905c-a31b-44cd-b121-5182f860de2f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:03.456169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395f397-66a7-483b-8d36-ab9eecc9df89 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:03.553223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f7d6b-1c60-4cde-b9c1-805fc8f5cdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:03.553949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f7d6b-1c60-4cde-b9c1-805fc8f5cdb8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:03.554225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f7d6b-1c60-4cde-b9c1-805fc8f5cdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:03.554497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f7d6b-1c60-4cde-b9c1-805fc8f5cdb8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:03.555178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f7d6b-1c60-4cde-b9c1-805fc8f5cdb8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:03.556559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f7d6b-1c60-4cde-b9c1-805fc8f5cdb8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-763f7d6b-1c60-4cde-b9c1-805fc8f5cdb8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:03.592276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:03.593271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:42:03.594117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:03.595888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 388/771] 10.222.0.79 () {56 vars in 1129 bytes} [Fri May 13 13:42:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:03.608313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f7d6b-1c60-4cde-b9c1-805fc8f5cdb8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:03.671489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2905c-a31b-44cd-b121-5182f860de2f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:03.741942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86552d-e5f7-4b97-9dc9-16a092ed8c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:03.744097 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86552d-e5f7-4b97-9dc9-16a092ed8c3d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:03.746374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86552d-e5f7-4b97-9dc9-16a092ed8c3d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:03.748552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86552d-e5f7-4b97-9dc9-16a092ed8c3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:03.776171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86552d-e5f7-4b97-9dc9-16a092ed8c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-cf86552d-e5f7-4b97-9dc9-16a092ed8c3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:03.785817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:03.786873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:03.787711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:03.790616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 384/772] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:42:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:03.799897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86552d-e5f7-4b97-9dc9-16a092ed8c3d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:03.865306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f7d6b-1c60-4cde-b9c1-805fc8f5cdb8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:04.000528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b865472-073b-4448-b55a-966a076bca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:04.003444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b865472-073b-4448-b55a-966a076bca9d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:04.009526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b865472-073b-4448-b55a-966a076bca9d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:04.010410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b865472-073b-4448-b55a-966a076bca9d None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:04.012137 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b865472-073b-4448-b55a-966a076bca9d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:04.013001 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b865472-073b-4448-b55a-966a076bca9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:04.018072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b865472-073b-4448-b55a-966a076bca9d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:04.037217 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 389/773] 10.222.0.79 () {56 vars in 1124 bytes} [Fri May 13 13:42:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:04.039346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b865472-073b-4448-b55a-966a076bca9d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:04.101970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86552d-e5f7-4b97-9dc9-16a092ed8c3d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:04.202650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da329f7-f96d-42be-9a96-c9029607779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:04.207596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da329f7-f96d-42be-9a96-c9029607779c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:04.208234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da329f7-f96d-42be-9a96-c9029607779c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:04.208741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da329f7-f96d-42be-9a96-c9029607779c None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:04.209736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da329f7-f96d-42be-9a96-c9029607779c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:04.210256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da329f7-f96d-42be-9a96-c9029607779c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:04.212873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da329f7-f96d-42be-9a96-c9029607779c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:04.243864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 385/774] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:42:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:04.246562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc059a-4bc0-4858-96db-9981ab06a0b6 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:04.247536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc059a-4bc0-4858-96db-9981ab06a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:04.248429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc059a-4bc0-4858-96db-9981ab06a0b6 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:04.252126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 386/775] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:04.260685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc059a-4bc0-4858-96db-9981ab06a0b6 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:04.278574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b865472-073b-4448-b55a-966a076bca9d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:04.348918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577b5d8-6684-45a8-92f0-8e58ad8a05e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:04.350482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b5d8-6684-45a8-92f0-8e58ad8a05e7 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:04.351117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b5d8-6684-45a8-92f0-8e58ad8a05e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:04.351626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577b5d8-6684-45a8-92f0-8e58ad8a05e7 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:04.352789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577b5d8-6684-45a8-92f0-8e58ad8a05e7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:04.353476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577b5d8-6684-45a8-92f0-8e58ad8a05e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:04.355834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577b5d8-6684-45a8-92f0-8e58ad8a05e7 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:04.365614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 390/776] 10.222.0.79 () {56 vars in 1124 bytes} [Fri May 13 13:42:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:04.367493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f12172-7214-4646-a9a3-8d8281f8556e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:04.368029 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f12172-7214-4646-a9a3-8d8281f8556e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:04.368511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f12172-7214-4646-a9a3-8d8281f8556e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:04.389208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f1f12172-7214-4646-a9a3-8d8281f8556e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:04.391230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f1f12172-7214-4646-a9a3-8d8281f8556e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:04.440666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc059a-4bc0-4858-96db-9981ab06a0b6 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:04.502651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdb2c7-cf19-4f70-9ee4-99caa0f8f970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:04.503411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdb2c7-cf19-4f70-9ee4-99caa0f8f970 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:04.504365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdb2c7-cf19-4f70-9ee4-99caa0f8f970 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:04.504665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbdb2c7-cf19-4f70-9ee4-99caa0f8f970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:04.513110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 391/777] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:04.515350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f12172-7214-4646-a9a3-8d8281f8556e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:04.530341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdb2c7-cf19-4f70-9ee4-99caa0f8f970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:04.532108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbdb2c7-cf19-4f70-9ee4-99caa0f8f970 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-3bbdb2c7-cf19-4f70-9ee4-99caa0f8f970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:04.563858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:04.565967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:04.567814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:04.567814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:04.567814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:04.567814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:04.567814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 387/778] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:04.567814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdb2c7-cf19-4f70-9ee4-99caa0f8f970 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:04.689325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f12172-7214-4646-a9a3-8d8281f8556e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:04.728391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbdb2c7-cf19-4f70-9ee4-99caa0f8f970 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:04.732541 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e437a4-e169-4cb2-81e5-90ae5fcd72ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:04.733222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e437a4-e169-4cb2-81e5-90ae5fcd72ec None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:04.733498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e437a4-e169-4cb2-81e5-90ae5fcd72ec None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:04.733912 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e437a4-e169-4cb2-81e5-90ae5fcd72ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:04.734980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e437a4-e169-4cb2-81e5-90ae5fcd72ec None admin] RBAC: Adding query filter params (interface=public, service_id=17676539ce0244c1be8b22767b7fc3b1, region_id=RegionOne) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:04.735525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e437a4-e169-4cb2-81e5-90ae5fcd72ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:04.737993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e437a4-e169-4cb2-81e5-90ae5fcd72ec None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:04.749279 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 392/779] 10.222.0.79 () {56 vars in 1251 bytes} [Fri May 13 13:42:04 2022] GET /identity/v3/endpoints?service_id=17676539ce0244c1be8b22767b7fc3b1&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:04.750151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e437a4-e169-4cb2-81e5-90ae5fcd72ec None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:04.786025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce2572-43a1-4af0-8dbe-7ea9af6df48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:04.786752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce2572-43a1-4af0-8dbe-7ea9af6df48e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:04.787036 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce2572-43a1-4af0-8dbe-7ea9af6df48e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:04.787304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce2572-43a1-4af0-8dbe-7ea9af6df48e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:04.799917 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce2572-43a1-4af0-8dbe-7ea9af6df48e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-55ce2572-43a1-4af0-8dbe-7ea9af6df48e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:04.814866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:04.817195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:04.818601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 388/780] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:42:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:04.821747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319db185-c540-40c5-8eb5-479f1166ff38 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:04.821747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319db185-c540-40c5-8eb5-479f1166ff38 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:04.821747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319db185-c540-40c5-8eb5-479f1166ff38 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:04.826536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 389/781] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:04.826536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319db185-c540-40c5-8eb5-479f1166ff38 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:04.924540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e437a4-e169-4cb2-81e5-90ae5fcd72ec None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:04.994899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee9953-5f63-4341-a973-e7dfef9e5fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:04.996005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee9953-5f63-4341-a973-e7dfef9e5fe0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:04.996448 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee9953-5f63-4341-a973-e7dfef9e5fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:04.996899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee9953-5f63-4341-a973-e7dfef9e5fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:04.998224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee9953-5f63-4341-a973-e7dfef9e5fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:04.998682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee9953-5f63-4341-a973-e7dfef9e5fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:05.001300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee9953-5f63-4341-a973-e7dfef9e5fe0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:05.026793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 393/782] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:05.029133 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee9953-5f63-4341-a973-e7dfef9e5fe0 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:05.029957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319db185-c540-40c5-8eb5-479f1166ff38 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:05.088712 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5d72d-1d58-478e-8ed4-5edde89f89c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:05.089654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5d72d-1d58-478e-8ed4-5edde89f89c4 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:05.090142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5d72d-1d58-478e-8ed4-5edde89f89c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:05.090620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5d72d-1d58-478e-8ed4-5edde89f89c4 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:05.092010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5d72d-1d58-478e-8ed4-5edde89f89c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:05.094631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5d72d-1d58-478e-8ed4-5edde89f89c4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:05.114747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 390/783] 10.222.0.79 () {56 vars in 1087 bytes} [Fri May 13 13:42:04 2022] GET /identity/v3/services => generated 772 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:05.114747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4f246-0708-4442-8186-78874c0b0a55 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:05.114747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4f246-0708-4442-8186-78874c0b0a55 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:05.114747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4f246-0708-4442-8186-78874c0b0a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:05.137480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-6fb4f246-0708-4442-8186-78874c0b0a55 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:05.138937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6fb4f246-0708-4442-8186-78874c0b0a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:05.224681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee9953-5f63-4341-a973-e7dfef9e5fe0 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:05.277135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 391/784] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:05.278883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4f246-0708-4442-8186-78874c0b0a55 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:05.287886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34a142-25a5-43ca-bc38-6a622cd7fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:05.288887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34a142-25a5-43ca-bc38-6a622cd7fe1d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:05.289287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34a142-25a5-43ca-bc38-6a622cd7fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:05.289680 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34a142-25a5-43ca-bc38-6a622cd7fe1d None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:05.290874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34a142-25a5-43ca-bc38-6a622cd7fe1d None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:05.291339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34a142-25a5-43ca-bc38-6a622cd7fe1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:05.293574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34a142-25a5-43ca-bc38-6a622cd7fe1d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:05.307620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 394/785] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:42:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:05.313095 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34a142-25a5-43ca-bc38-6a622cd7fe1d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:05.449657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4f246-0708-4442-8186-78874c0b0a55 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:05.515321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34a142-25a5-43ca-bc38-6a622cd7fe1d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:05.517486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612a287-b227-418e-ad56-dabe91286c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:05.518971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612a287-b227-418e-ad56-dabe91286c0d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:05.519612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612a287-b227-418e-ad56-dabe91286c0d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:05.520088 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612a287-b227-418e-ad56-dabe91286c0d None admin] PATH_INFO: `/v3/users/swift` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:05.532345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612a287-b227-418e-ad56-dabe91286c0d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:05.535357 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612a287-b227-418e-ad56-dabe91286c0d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-3612a287-b227-418e-ad56-dabe91286c0d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:05.559288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:05.560675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:05.561841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:05.561841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:05.561841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:05.561841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:05.561841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:05.561841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:05.561841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:05.561841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:05.561841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:42:05.561841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:05.562500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49af70d-114d-46a3-8cb0-f24aefb406c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:05.562603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 392/786] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:42:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:05.562678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49af70d-114d-46a3-8cb0-f24aefb406c2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:05.564605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612a287-b227-418e-ad56-dabe91286c0d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:05.569656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49af70d-114d-46a3-8cb0-f24aefb406c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:05.570190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49af70d-114d-46a3-8cb0-f24aefb406c2 None admin] PATH_INFO: `/v3/services/network` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:05.572777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49af70d-114d-46a3-8cb0-f24aefb406c2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:05.573448 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49af70d-114d-46a3-8cb0-f24aefb406c2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-c49af70d-114d-46a3-8cb0-f24aefb406c2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:05.599259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:05.601951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:42:05.604166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:05.614865 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 395/787] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:05 2022] GET /identity/v3/services/network => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:05.614865 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49af70d-114d-46a3-8cb0-f24aefb406c2 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:05.734408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612a287-b227-418e-ad56-dabe91286c0d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:05.769964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49af70d-114d-46a3-8cb0-f24aefb406c2 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:05.782304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e2e15-1a5b-4319-871a-2633313d79d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:05.783187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2e15-1a5b-4319-871a-2633313d79d0 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:05.783467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2e15-1a5b-4319-871a-2633313d79d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:05.783740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2e15-1a5b-4319-871a-2633313d79d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:05.797960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e2e15-1a5b-4319-871a-2633313d79d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:05.799554 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e2e15-1a5b-4319-871a-2633313d79d0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:05.813924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec56b-6a2f-49d3-ac46-603a4253567d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:05.814601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec56b-6a2f-49d3-ac46-603a4253567d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:05.814944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec56b-6a2f-49d3-ac46-603a4253567d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:05.815224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec56b-6a2f-49d3-ac46-603a4253567d None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:05.817478 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ec56b-6a2f-49d3-ac46-603a4253567d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:05.818454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ec56b-6a2f-49d3-ac46-603a4253567d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:05.820492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ec56b-6a2f-49d3-ac46-603a4253567d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-352e2e15-1a5b-4319-871a-2633313d79d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:05.821691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:05.825488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:05.828629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:05.828629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:05.828629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:05.828629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:05.829242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 393/788] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:05.838786 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e2e15-1a5b-4319-871a-2633313d79d0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:05.860227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 396/789] 10.222.0.79 () {56 vars in 1135 bytes} [Fri May 13 13:42:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:05.861898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec56b-6a2f-49d3-ac46-603a4253567d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:05.981171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e2e15-1a5b-4319-871a-2633313d79d0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:06.036505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51446039-401c-40d7-89ad-493f3cc0dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:06.037081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51446039-401c-40d7-89ad-493f3cc0dd5f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.037360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51446039-401c-40d7-89ad-493f3cc0dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.037635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51446039-401c-40d7-89ad-493f3cc0dd5f None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.038441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51446039-401c-40d7-89ad-493f3cc0dd5f None admin] RBAC: Adding query filter params (name=network) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:06.039130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51446039-401c-40d7-89ad-493f3cc0dd5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:06.040201 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51446039-401c-40d7-89ad-493f3cc0dd5f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:06.050887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 394/790] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:06.050887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51446039-401c-40d7-89ad-493f3cc0dd5f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:06.077098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec56b-6a2f-49d3-ac46-603a4253567d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:06.133137 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd83ed3-e595-45e4-8198-975d46aa8feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:06.133706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd83ed3-e595-45e4-8198-975d46aa8feb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.133984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd83ed3-e595-45e4-8198-975d46aa8feb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.134257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd83ed3-e595-45e4-8198-975d46aa8feb None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.135016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd83ed3-e595-45e4-8198-975d46aa8feb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:06.135319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd83ed3-e595-45e4-8198-975d46aa8feb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:06.136742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd83ed3-e595-45e4-8198-975d46aa8feb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:06.154117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 397/791] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:06.155011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd83ed3-e595-45e4-8198-975d46aa8feb None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:06.166965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51446039-401c-40d7-89ad-493f3cc0dd5f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:06.207429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb77ae-7fe5-4c43-8feb-ecd0bcf5fb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:06.209366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb77ae-7fe5-4c43-8feb-ecd0bcf5fb0b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.209832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb77ae-7fe5-4c43-8feb-ecd0bcf5fb0b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.210255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb77ae-7fe5-4c43-8feb-ecd0bcf5fb0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.226149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb77ae-7fe5-4c43-8feb-ecd0bcf5fb0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:06.227664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb77ae-7fe5-4c43-8feb-ecd0bcf5fb0b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-d4bb77ae-7fe5-4c43-8feb-ecd0bcf5fb0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:06.247850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:06.248971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:06.249881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:06.249881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:06.249881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:06.249881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:06.249881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 395/792] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:06.251740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45448c77-a47a-470e-9e9f-139c162c201e None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.251740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45448c77-a47a-470e-9e9f-139c162c201e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.251740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45448c77-a47a-470e-9e9f-139c162c201e None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.258232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 396/793] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:06.258232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45448c77-a47a-470e-9e9f-139c162c201e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:06.293308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd83ed3-e595-45e4-8198-975d46aa8feb None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:06.333675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a04b41-76f2-4d5c-8362-988ed48efdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:06.334342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a04b41-76f2-4d5c-8362-988ed48efdcd None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.334618 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a04b41-76f2-4d5c-8362-988ed48efdcd None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.334921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a04b41-76f2-4d5c-8362-988ed48efdcd None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.335623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a04b41-76f2-4d5c-8362-988ed48efdcd None admin] RBAC: Adding query filter params (type=network) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:06.335919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a04b41-76f2-4d5c-8362-988ed48efdcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:06.337302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a04b41-76f2-4d5c-8362-988ed48efdcd None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:06.350186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 398/794] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:06.350186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a04b41-76f2-4d5c-8362-988ed48efdcd None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:06.384739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45448c77-a47a-470e-9e9f-139c162c201e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:06.425776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ce291-2da4-4c3a-971c-16530e74b655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:06.426580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ce291-2da4-4c3a-971c-16530e74b655 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.426887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ce291-2da4-4c3a-971c-16530e74b655 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.427162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ce291-2da4-4c3a-971c-16530e74b655 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.434791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ce291-2da4-4c3a-971c-16530e74b655 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:06.436258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ce291-2da4-4c3a-971c-16530e74b655 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:06.449007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a04b41-76f2-4d5c-8362-988ed48efdcd None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-ab9ce291-2da4-4c3a-971c-16530e74b655 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:06.452142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:06.453296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:06.454271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 397/795] 10.222.0.79 () {56 vars in 1117 bytes} [Fri May 13 13:42:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:06.455915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c83d1-7f3a-41bc-9604-e05ab9b2b915 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.455915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c83d1-7f3a-41bc-9604-e05ab9b2b915 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.455915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c83d1-7f3a-41bc-9604-e05ab9b2b915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.478553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b65c83d1-7f3a-41bc-9604-e05ab9b2b915 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:06.480061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b65c83d1-7f3a-41bc-9604-e05ab9b2b915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:06.489728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b90503-8b1d-4e42-b8ca-17e1ee3061d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:06.490011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b90503-8b1d-4e42-b8ca-17e1ee3061d5 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.490322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b90503-8b1d-4e42-b8ca-17e1ee3061d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.490532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b90503-8b1d-4e42-b8ca-17e1ee3061d5 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.491282 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b90503-8b1d-4e42-b8ca-17e1ee3061d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:06.491593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b90503-8b1d-4e42-b8ca-17e1ee3061d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:06.492976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b90503-8b1d-4e42-b8ca-17e1ee3061d5 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:06.505619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 399/796] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:06.506467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b90503-8b1d-4e42-b8ca-17e1ee3061d5 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:06.573741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 398/797] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:06.575373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c83d1-7f3a-41bc-9604-e05ab9b2b915 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:06.642178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b90503-8b1d-4e42-b8ca-17e1ee3061d5 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:06.678810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6122f-1ef7-4f9c-9817-087cc2ee7381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:06.679659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6122f-1ef7-4f9c-9817-087cc2ee7381 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.680019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6122f-1ef7-4f9c-9817-087cc2ee7381 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.680362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6122f-1ef7-4f9c-9817-087cc2ee7381 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.681236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6122f-1ef7-4f9c-9817-087cc2ee7381 None admin] RBAC: Adding query filter params (interface=public, service_id=bba820b49acf4ccc92d3b4c88385595d, region_id=RegionOne) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:06.681555 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6122f-1ef7-4f9c-9817-087cc2ee7381 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:06.682993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6122f-1ef7-4f9c-9817-087cc2ee7381 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:06.693430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 400/798] 10.222.0.79 () {56 vars in 1251 bytes} [Fri May 13 13:42:06 2022] GET /identity/v3/endpoints?service_id=bba820b49acf4ccc92d3b4c88385595d&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:06.693430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6122f-1ef7-4f9c-9817-087cc2ee7381 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:06.705900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c83d1-7f3a-41bc-9604-e05ab9b2b915 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:06.749322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d33472-451c-4af1-8749-c9db72a93574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:06.750604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d33472-451c-4af1-8749-c9db72a93574 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.751118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d33472-451c-4af1-8749-c9db72a93574 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.751792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d33472-451c-4af1-8749-c9db72a93574 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.753607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d33472-451c-4af1-8749-c9db72a93574 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:06.754140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d33472-451c-4af1-8749-c9db72a93574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:06.756903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d33472-451c-4af1-8749-c9db72a93574 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:06.784364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 399/799] 10.222.0.79 () {56 vars in 1149 bytes} [Fri May 13 13:42:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:06.785818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d33472-451c-4af1-8749-c9db72a93574 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:06.819960 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6122f-1ef7-4f9c-9817-087cc2ee7381 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:06.859869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32626359-fe9c-4309-a534-1f0a1e4ba717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:06.860620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32626359-fe9c-4309-a534-1f0a1e4ba717 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.860906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32626359-fe9c-4309-a534-1f0a1e4ba717 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.861202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32626359-fe9c-4309-a534-1f0a1e4ba717 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.869584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32626359-fe9c-4309-a534-1f0a1e4ba717 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:06.871142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32626359-fe9c-4309-a534-1f0a1e4ba717 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-32626359-fe9c-4309-a534-1f0a1e4ba717 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:06.885712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:06.886764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:06.887695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:06.887695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:06.887695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:06.887695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:06.887695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:06.887695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:06.887695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:06.887695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:06.887695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 401/800] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:06.888279 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32626359-fe9c-4309-a534-1f0a1e4ba717 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:06.931894 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d33472-451c-4af1-8749-c9db72a93574 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:06.980912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365b6b4-2c59-498b-b6ba-622b73379395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:06.981924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365b6b4-2c59-498b-b6ba-622b73379395 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:06.982394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365b6b4-2c59-498b-b6ba-622b73379395 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:06.982817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365b6b4-2c59-498b-b6ba-622b73379395 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:06.997321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365b6b4-2c59-498b-b6ba-622b73379395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-3365b6b4-2c59-498b-b6ba-622b73379395 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:07.005308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:07.007007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 400/801] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:42:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:07.007899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365b6b4-2c59-498b-b6ba-622b73379395 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:07.040538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32626359-fe9c-4309-a534-1f0a1e4ba717 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:07.101054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139335c6-ba31-4a91-ae36-2dd1ec3f9631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:07.102077 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139335c6-ba31-4a91-ae36-2dd1ec3f9631 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:07.102237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139335c6-ba31-4a91-ae36-2dd1ec3f9631 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:07.102545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139335c6-ba31-4a91-ae36-2dd1ec3f9631 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:07.103787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139335c6-ba31-4a91-ae36-2dd1ec3f9631 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:07.105294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139335c6-ba31-4a91-ae36-2dd1ec3f9631 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:07.116031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 402/802] 10.222.0.79 () {56 vars in 1087 bytes} [Fri May 13 13:42:06 2022] GET /identity/v3/services => generated 772 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:07.117928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139335c6-ba31-4a91-ae36-2dd1ec3f9631 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:07.158876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365b6b4-2c59-498b-b6ba-622b73379395 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:07.214195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19434f44-2110-44c2-8511-f6e6ac1a3e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:07.216473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19434f44-2110-44c2-8511-f6e6ac1a3e75 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:07.217243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19434f44-2110-44c2-8511-f6e6ac1a3e75 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:07.218173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19434f44-2110-44c2-8511-f6e6ac1a3e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:07.237891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19434f44-2110-44c2-8511-f6e6ac1a3e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:07.241497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19434f44-2110-44c2-8511-f6e6ac1a3e75 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-19434f44-2110-44c2-8511-f6e6ac1a3e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:07.272614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:07.273606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:07.274590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:07.274590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:07.274590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:07.274590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:07.277847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 401/803] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:07.279193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19434f44-2110-44c2-8511-f6e6ac1a3e75 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:07.329847 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139335c6-ba31-4a91-ae36-2dd1ec3f9631 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:07.403481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735960a4-2816-4ff5-bce9-9759b6acea1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:07.404842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735960a4-2816-4ff5-bce9-9759b6acea1e None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:07.405098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735960a4-2816-4ff5-bce9-9759b6acea1e None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:07.405528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735960a4-2816-4ff5-bce9-9759b6acea1e None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:07.407245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735960a4-2816-4ff5-bce9-9759b6acea1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:07.407806 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735960a4-2816-4ff5-bce9-9759b6acea1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:07.410555 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735960a4-2816-4ff5-bce9-9759b6acea1e None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:07.435903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 403/804] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:07.442436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735960a4-2816-4ff5-bce9-9759b6acea1e None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:07.480233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19434f44-2110-44c2-8511-f6e6ac1a3e75 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:07.553140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0c9e4-fb87-4a00-a6cf-84c1f4e928bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:07.554752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0c9e4-fb87-4a00-a6cf-84c1f4e928bd None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:07.555524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0c9e4-fb87-4a00-a6cf-84c1f4e928bd None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:07.556269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0c9e4-fb87-4a00-a6cf-84c1f4e928bd None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:07.557885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0c9e4-fb87-4a00-a6cf-84c1f4e928bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:07.558702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0c9e4-fb87-4a00-a6cf-84c1f4e928bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:07.561667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0c9e4-fb87-4a00-a6cf-84c1f4e928bd None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:07.579451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 402/805] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:42:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:07.581440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0c9e4-fb87-4a00-a6cf-84c1f4e928bd None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:07.663212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735960a4-2816-4ff5-bce9-9759b6acea1e None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:07.719062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e9a6c-968e-427c-8e35-ccc77e1bb291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:07.719552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e9a6c-968e-427c-8e35-ccc77e1bb291 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:07.719875 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e9a6c-968e-427c-8e35-ccc77e1bb291 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:07.720308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e9a6c-968e-427c-8e35-ccc77e1bb291 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:07.721162 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e9a6c-968e-427c-8e35-ccc77e1bb291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:07.721465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e9a6c-968e-427c-8e35-ccc77e1bb291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:07.723019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e9a6c-968e-427c-8e35-ccc77e1bb291 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:07.740485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 404/806] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:07.741962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e9a6c-968e-427c-8e35-ccc77e1bb291 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:07.781237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0c9e4-fb87-4a00-a6cf-84c1f4e928bd None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:07.858064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306db5a0-4602-4f68-aebe-ca255691eb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:07.860170 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306db5a0-4602-4f68-aebe-ca255691eb89 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:07.860998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306db5a0-4602-4f68-aebe-ca255691eb89 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:07.861694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306db5a0-4602-4f68-aebe-ca255691eb89 None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/4139cfe906bd4c7d94226ab589577627/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:07.865518 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306db5a0-4602-4f68-aebe-ca255691eb89 None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=4139cfe906bd4c7d94226ab589577627, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:07.868427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306db5a0-4602-4f68-aebe-ca255691eb89 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:07.869449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e9a6c-968e-427c-8e35-ccc77e1bb291 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:07.902746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 403/807] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:42:07 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/4139cfe906bd4c7d94226ab589577627/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:42:07.902746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306db5a0-4602-4f68-aebe-ca255691eb89 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:07.968734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87183b-f49e-458b-afe6-4e1cabbf5793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:07.971303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87183b-f49e-458b-afe6-4e1cabbf5793 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:07.974131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87183b-f49e-458b-afe6-4e1cabbf5793 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:07.974741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87183b-f49e-458b-afe6-4e1cabbf5793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:08.015914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87183b-f49e-458b-afe6-4e1cabbf5793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:08.017673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87183b-f49e-458b-afe6-4e1cabbf5793 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-8d87183b-f49e-458b-afe6-4e1cabbf5793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:08.103709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:08.106240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:08.108181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:08.108181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:08.108181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:08.108181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:08.112819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 405/808] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:08.121472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8d7ed-ccf1-4b91-b750-b74003f8f586 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:08.122325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8d7ed-ccf1-4b91-b750-b74003f8f586 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:08.123144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8d7ed-ccf1-4b91-b750-b74003f8f586 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:08.127058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 406/809] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:08.129273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8d7ed-ccf1-4b91-b750-b74003f8f586 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:08.181535 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306db5a0-4602-4f68-aebe-ca255691eb89 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:08.234415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3272857-2305-4ed1-bfae-9b635edd960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:08.235208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3272857-2305-4ed1-bfae-9b635edd960d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:08.235487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3272857-2305-4ed1-bfae-9b635edd960d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:08.235767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3272857-2305-4ed1-bfae-9b635edd960d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:08.246007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3272857-2305-4ed1-bfae-9b635edd960d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:08.247739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3272857-2305-4ed1-bfae-9b635edd960d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-f3272857-2305-4ed1-bfae-9b635edd960d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:08.267618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:08.268660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:08.269598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:08.269598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:08.269598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:08.269598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:08.269598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:08.269598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:08.269598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:08.269598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:08.269598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 404/810] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:08.270622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb4d2-054b-4f86-b24b-8ea5e8462541 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:08.270900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb4d2-054b-4f86-b24b-8ea5e8462541 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:08.271178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb4d2-054b-4f86-b24b-8ea5e8462541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:08.297041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8d7ed-ccf1-4b91-b750-b74003f8f586 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:08.302473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-2a3bb4d2-054b-4f86-b24b-8ea5e8462541 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:08.304719 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2a3bb4d2-054b-4f86-b24b-8ea5e8462541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:08.335651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61796775-0ea7-473b-8741-2a330b97d251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:08.336344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61796775-0ea7-473b-8741-2a330b97d251 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:08.336629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61796775-0ea7-473b-8741-2a330b97d251 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:08.336899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61796775-0ea7-473b-8741-2a330b97d251 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:08.337638 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61796775-0ea7-473b-8741-2a330b97d251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:08.337942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61796775-0ea7-473b-8741-2a330b97d251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:08.339392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61796775-0ea7-473b-8741-2a330b97d251 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:08.358194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 407/811] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:08.359456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61796775-0ea7-473b-8741-2a330b97d251 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:08.401073 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 405/812] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:08.402305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bb4d2-054b-4f86-b24b-8ea5e8462541 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:08.504667 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61796775-0ea7-473b-8741-2a330b97d251 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:08.537059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bb4d2-054b-4f86-b24b-8ea5e8462541 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:08.543371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad484c6d-e836-482c-aac4-8efc625f46cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:08.544051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad484c6d-e836-482c-aac4-8efc625f46cc None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:08.544334 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad484c6d-e836-482c-aac4-8efc625f46cc None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:08.544610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad484c6d-e836-482c-aac4-8efc625f46cc None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:08.545420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad484c6d-e836-482c-aac4-8efc625f46cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:08.545727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad484c6d-e836-482c-aac4-8efc625f46cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:08.547252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad484c6d-e836-482c-aac4-8efc625f46cc None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:08.562453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 408/813] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:08.563798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad484c6d-e836-482c-aac4-8efc625f46cc None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:08.580847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf691012-8a03-40be-8b60-bbe03d134c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:08.582687 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf691012-8a03-40be-8b60-bbe03d134c4e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:08.583319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf691012-8a03-40be-8b60-bbe03d134c4e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:08.583862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf691012-8a03-40be-8b60-bbe03d134c4e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:08.593693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf691012-8a03-40be-8b60-bbe03d134c4e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:08.596313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf691012-8a03-40be-8b60-bbe03d134c4e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-cf691012-8a03-40be-8b60-bbe03d134c4e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:08.617064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:08.618091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:08.619081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:08.619081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:08.619081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:08.619081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:08.619081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:08.619081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:08.619081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:08.619081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:08.619081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:42:08.619081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:08.621074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 406/814] 10.222.0.79 () {56 vars in 1093 bytes} [Fri May 13 13:42:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:08.624201 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf691012-8a03-40be-8b60-bbe03d134c4e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:08.752429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad484c6d-e836-482c-aac4-8efc625f46cc None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:08.800198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf691012-8a03-40be-8b60-bbe03d134c4e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:08.808171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48201de-c0a8-471f-94d4-4e2756fdae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:08.810308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48201de-c0a8-471f-94d4-4e2756fdae27 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:08.810923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48201de-c0a8-471f-94d4-4e2756fdae27 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:08.811449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48201de-c0a8-471f-94d4-4e2756fdae27 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:08.812879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48201de-c0a8-471f-94d4-4e2756fdae27 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:08.814763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48201de-c0a8-471f-94d4-4e2756fdae27 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-d48201de-c0a8-471f-94d4-4e2756fdae27 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:08.833588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:08.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:42:08.835609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:08.838618 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 409/815] 10.222.0.79 () {56 vars in 1129 bytes} [Fri May 13 13:42:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:08.839627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48201de-c0a8-471f-94d4-4e2756fdae27 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:08.857306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804dae8-36c7-4bdf-9c8f-628da1559488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:08.858379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804dae8-36c7-4bdf-9c8f-628da1559488 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:08.858783 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804dae8-36c7-4bdf-9c8f-628da1559488 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:08.859253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804dae8-36c7-4bdf-9c8f-628da1559488 None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/598c323e8ed5455cbe4a568b360d8eab/roles/879ba54490a944d6abcf2ac5e2f1bea9` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:08.875797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804dae8-36c7-4bdf-9c8f-628da1559488 None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=598c323e8ed5455cbe4a568b360d8eab, role_id=879ba54490a944d6abcf2ac5e2f1bea9)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:08.877422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804dae8-36c7-4bdf-9c8f-628da1559488 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:08.893703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 407/816] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:42:08 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/598c323e8ed5455cbe4a568b360d8eab/roles/879ba54490a944d6abcf2ac5e2f1bea9 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:42:08.895173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804dae8-36c7-4bdf-9c8f-628da1559488 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:08.999443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48201de-c0a8-471f-94d4-4e2756fdae27 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:09.068077 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839384d-501f-4923-9b17-0b0133e7cc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:09.068707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839384d-501f-4923-9b17-0b0133e7cc0f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:09.068989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839384d-501f-4923-9b17-0b0133e7cc0f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:09.069263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839384d-501f-4923-9b17-0b0133e7cc0f None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:09.070111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839384d-501f-4923-9b17-0b0133e7cc0f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:09.070420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839384d-501f-4923-9b17-0b0133e7cc0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:09.071914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839384d-501f-4923-9b17-0b0133e7cc0f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:09.105806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804dae8-36c7-4bdf-9c8f-628da1559488 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:09.111969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 410/817] 10.222.0.79 () {56 vars in 1133 bytes} [Fri May 13 13:42:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:09.118931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839384d-501f-4923-9b17-0b0133e7cc0f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:09.181832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a826b-24cc-4613-87f1-42676e9dc2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:09.182510 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a826b-24cc-4613-87f1-42676e9dc2e9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:09.182810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a826b-24cc-4613-87f1-42676e9dc2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:09.183091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a826b-24cc-4613-87f1-42676e9dc2e9 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:09.183811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a826b-24cc-4613-87f1-42676e9dc2e9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:09.184118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a826b-24cc-4613-87f1-42676e9dc2e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:09.185543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a826b-24cc-4613-87f1-42676e9dc2e9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:09.191003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 408/818] 10.222.0.79 () {56 vars in 1124 bytes} [Fri May 13 13:42:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:09.198822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a826b-24cc-4613-87f1-42676e9dc2e9 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:09.302235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839384d-501f-4923-9b17-0b0133e7cc0f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:09.344072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644a5e7-76b3-47c4-a3f7-de611fcdc66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:09.344644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644a5e7-76b3-47c4-a3f7-de611fcdc66f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:09.344928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644a5e7-76b3-47c4-a3f7-de611fcdc66f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:09.345204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644a5e7-76b3-47c4-a3f7-de611fcdc66f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:09.346387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a826b-24cc-4613-87f1-42676e9dc2e9 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:09.390512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2392eb1-9c86-4d51-a37f-33bdfe0adf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:09.391913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2392eb1-9c86-4d51-a37f-33bdfe0adf18 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:09.392422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2392eb1-9c86-4d51-a37f-33bdfe0adf18 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:09.392923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2392eb1-9c86-4d51-a37f-33bdfe0adf18 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:09.394580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2392eb1-9c86-4d51-a37f-33bdfe0adf18 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:09.395133 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2392eb1-9c86-4d51-a37f-33bdfe0adf18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:09.397493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2392eb1-9c86-4d51-a37f-33bdfe0adf18 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:09.404574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 409/819] 10.222.0.79 () {56 vars in 1124 bytes} [Fri May 13 13:42:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:09.417210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2392eb1-9c86-4d51-a37f-33bdfe0adf18 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:09.487627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644a5e7-76b3-47c4-a3f7-de611fcdc66f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:09.489184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644a5e7-76b3-47c4-a3f7-de611fcdc66f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:09.564560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2392eb1-9c86-4d51-a37f-33bdfe0adf18 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:09.616319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e18ce-a2bc-4c82-91d3-0680cfa53be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:09.617484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e18ce-a2bc-4c82-91d3-0680cfa53be2 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:09.618296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e18ce-a2bc-4c82-91d3-0680cfa53be2 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:09.618818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e18ce-a2bc-4c82-91d3-0680cfa53be2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:09.619795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e18ce-a2bc-4c82-91d3-0680cfa53be2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:09.622039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e18ce-a2bc-4c82-91d3-0680cfa53be2 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:09.654323 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 410/820] 10.222.0.79 () {60 vars in 1144 bytes} [Fri May 13 13:42:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:42:09.742849 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 411/821] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:42:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:42:09.751477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e18ce-a2bc-4c82-91d3-0680cfa53be2 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:09.989221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e18ce-a2bc-4c82-91d3-0680cfa53be2 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:10.085755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b1fe4-eaeb-4a07-8f87-105eb8bb2032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:10.087180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b1fe4-eaeb-4a07-8f87-105eb8bb2032 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:10.087663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b1fe4-eaeb-4a07-8f87-105eb8bb2032 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:10.088137 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b1fe4-eaeb-4a07-8f87-105eb8bb2032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:10.116650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b1fe4-eaeb-4a07-8f87-105eb8bb2032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:10.118868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b1fe4-eaeb-4a07-8f87-105eb8bb2032 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-d57b1fe4-eaeb-4a07-8f87-105eb8bb2032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:10.164736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:10.165733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:10.167112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:10.167112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:10.167112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:10.167112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:10.167839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 411/822] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:10.177036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644a5e7-76b3-47c4-a3f7-de611fcdc66f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:10.324580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644a5e7-76b3-47c4-a3f7-de611fcdc66f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:10.339864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a715b9-72d3-4255-a84f-432b407d786b None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:10.339864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a715b9-72d3-4255-a84f-432b407d786b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:10.339864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a715b9-72d3-4255-a84f-432b407d786b None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:10.340580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 412/823] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:10.346342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb377d-591c-40aa-8e24-2f41b4384116 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:10.346614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb377d-591c-40aa-8e24-2f41b4384116 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:10.347020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb377d-591c-40aa-8e24-2f41b4384116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:10.370254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-4fdb377d-591c-40aa-8e24-2f41b4384116 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:10.371545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4fdb377d-591c-40aa-8e24-2f41b4384116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:10.372597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b3efa-60c0-4248-8951-660ac9ed061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:10.373531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b3efa-60c0-4248-8951-660ac9ed061b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:10.374200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b3efa-60c0-4248-8951-660ac9ed061b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:10.375593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b3efa-60c0-4248-8951-660ac9ed061b None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:10.376571 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b3efa-60c0-4248-8951-660ac9ed061b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:10.377084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b3efa-60c0-4248-8951-660ac9ed061b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:10.378753 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b3efa-60c0-4248-8951-660ac9ed061b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:10.394582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 412/824] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:10.403239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b3efa-60c0-4248-8951-660ac9ed061b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:10.453536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 413/825] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:10.495324 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb377d-591c-40aa-8e24-2f41b4384116 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:10.530109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b3efa-60c0-4248-8951-660ac9ed061b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:10.569449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae3899-29ca-4026-94ee-6c08d179b75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:10.570687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3899-29ca-4026-94ee-6c08d179b75c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:10.571268 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3899-29ca-4026-94ee-6c08d179b75c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:10.571697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3899-29ca-4026-94ee-6c08d179b75c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:10.579967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae3899-29ca-4026-94ee-6c08d179b75c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:10.582452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae3899-29ca-4026-94ee-6c08d179b75c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-8fae3899-29ca-4026-94ee-6c08d179b75c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:10.603139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:10.604402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:10.605511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:10.605511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:10.605511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:10.605511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:10.605511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:10.605511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:10.605511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:10.605511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:10.609003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 413/826] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:10.619874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae3899-29ca-4026-94ee-6c08d179b75c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:10.716998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb377d-591c-40aa-8e24-2f41b4384116 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:10.808227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae3899-29ca-4026-94ee-6c08d179b75c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:10.811907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d69030-2f47-462f-bafa-068cb17140f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:10.813957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d69030-2f47-462f-bafa-068cb17140f9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:10.813957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d69030-2f47-462f-bafa-068cb17140f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:10.814422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d69030-2f47-462f-bafa-068cb17140f9 None admin] PATH_INFO: `/v3/services/network` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:10.815798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d69030-2f47-462f-bafa-068cb17140f9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:10.818500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d69030-2f47-462f-bafa-068cb17140f9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-29d69030-2f47-462f-bafa-068cb17140f9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:10.865679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:10.869151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:42:10.872462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:10.874671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 414/827] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:10 2022] GET /identity/v3/services/network => generated 88 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:10.874860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089692fb-e987-42fa-a767-141812563940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:10.875400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089692fb-e987-42fa-a767-141812563940 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:10.875887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089692fb-e987-42fa-a767-141812563940 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:10.876369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089692fb-e987-42fa-a767-141812563940 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:10.877893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089692fb-e987-42fa-a767-141812563940 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:10.878460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089692fb-e987-42fa-a767-141812563940 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:10.885192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089692fb-e987-42fa-a767-141812563940 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:10.888407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d69030-2f47-462f-bafa-068cb17140f9 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:10.906991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 414/828] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:10.916154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089692fb-e987-42fa-a767-141812563940 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:11.168879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089692fb-e987-42fa-a767-141812563940 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:11.214552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ee98b-4fe2-4ae8-852b-1754b538918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:11.216155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ee98b-4fe2-4ae8-852b-1754b538918a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:11.216666 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ee98b-4fe2-4ae8-852b-1754b538918a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:11.217136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ee98b-4fe2-4ae8-852b-1754b538918a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:11.218540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ee98b-4fe2-4ae8-852b-1754b538918a None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=24ad0dd9d265425087be2be8d245b7fc) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:11.219116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ee98b-4fe2-4ae8-852b-1754b538918a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:11.220784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ee98b-4fe2-4ae8-852b-1754b538918a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:11.229319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 415/829] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:42:10 2022] GET /identity/v3/role_assignments?user.id=24ad0dd9d265425087be2be8d245b7fc&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:11.329320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d69030-2f47-462f-bafa-068cb17140f9 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:11.403999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad4a13-1af7-4495-acc3-d27aa631f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:11.405147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4a13-1af7-4495-acc3-d27aa631f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:11.405637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4a13-1af7-4495-acc3-d27aa631f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:11.406126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4a13-1af7-4495-acc3-d27aa631f1d8 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:11.407502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad4a13-1af7-4495-acc3-d27aa631f1d8 None admin] RBAC: Adding query filter params (name=network) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:11.408054 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad4a13-1af7-4495-acc3-d27aa631f1d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:11.411274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad4a13-1af7-4495-acc3-d27aa631f1d8 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:11.420589 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 415/830] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:11.427544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ee98b-4fe2-4ae8-852b-1754b538918a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:11.429040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a49ffd-7a3f-42a5-955c-d5a796d65afa None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:11.429529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a49ffd-7a3f-42a5-955c-d5a796d65afa None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:11.430023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a49ffd-7a3f-42a5-955c-d5a796d65afa None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:11.433291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 416/831] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:11.442160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e615dcf-5118-4548-9a04-a0e2f6c86643 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:11.442655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e615dcf-5118-4548-9a04-a0e2f6c86643 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:11.443253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e615dcf-5118-4548-9a04-a0e2f6c86643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:11.473650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-3e615dcf-5118-4548-9a04-a0e2f6c86643 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:11.476349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-3e615dcf-5118-4548-9a04-a0e2f6c86643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:11.606122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 417/832] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:11.621262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ee98b-4fe2-4ae8-852b-1754b538918a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:11.650524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e615dcf-5118-4548-9a04-a0e2f6c86643 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:11.674328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd2586-8bb7-4d36-9ce8-a42753c2d6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:11.675320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd2586-8bb7-4d36-9ce8-a42753c2d6c4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:11.675777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd2586-8bb7-4d36-9ce8-a42753c2d6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:11.676208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd2586-8bb7-4d36-9ce8-a42753c2d6c4 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:11.677061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd2586-8bb7-4d36-9ce8-a42753c2d6c4 None admin] RBAC: Adding query filter params (type=network) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:11.677517 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd2586-8bb7-4d36-9ce8-a42753c2d6c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:11.679212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd2586-8bb7-4d36-9ce8-a42753c2d6c4 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:11.687151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 416/833] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:11.696254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd2586-8bb7-4d36-9ce8-a42753c2d6c4 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:11.833934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e615dcf-5118-4548-9a04-a0e2f6c86643 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:11.886305 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd2586-8bb7-4d36-9ce8-a42753c2d6c4 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:11.904525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82babb-cc79-4e9a-9f4d-ddb3585f5af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:11.906523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82babb-cc79-4e9a-9f4d-ddb3585f5af4 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:11.907490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82babb-cc79-4e9a-9f4d-ddb3585f5af4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:11.908129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82babb-cc79-4e9a-9f4d-ddb3585f5af4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:11.934503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82babb-cc79-4e9a-9f4d-ddb3585f5af4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:11.947029 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a614dd-4b7f-459e-bfb9-46c4c16adbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:11.948043 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a614dd-4b7f-459e-bfb9-46c4c16adbe5 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:11.948501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a614dd-4b7f-459e-bfb9-46c4c16adbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:11.948958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a614dd-4b7f-459e-bfb9-46c4c16adbe5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:11.950636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a614dd-4b7f-459e-bfb9-46c4c16adbe5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-7c82babb-cc79-4e9a-9f4d-ddb3585f5af4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:11.951761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:11.953821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:11.955625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:11.956979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a614dd-4b7f-459e-bfb9-46c4c16adbe5 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:11.959901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 418/834] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:11.969583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82babb-cc79-4e9a-9f4d-ddb3585f5af4 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:11.990464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 417/835] 10.222.0.79 () {60 vars in 1144 bytes} [Fri May 13 13:42:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:42:12.190509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82babb-cc79-4e9a-9f4d-ddb3585f5af4 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:12.265624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7f178-5e75-44c4-989f-f99498f89d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:12.266833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7f178-5e75-44c4-989f-f99498f89d18 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:12.267131 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7f178-5e75-44c4-989f-f99498f89d18 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:12.267416 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7f178-5e75-44c4-989f-f99498f89d18 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:12.268207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7f178-5e75-44c4-989f-f99498f89d18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:12.268515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7f178-5e75-44c4-989f-f99498f89d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:12.270006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7f178-5e75-44c4-989f-f99498f89d18 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:12.284162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 419/836] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:12.293727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a614dd-4b7f-459e-bfb9-46c4c16adbe5 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:12.492618 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a614dd-4b7f-459e-bfb9-46c4c16adbe5 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:12.561333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af075bca-b3f4-4a45-8a13-ddeddd2898af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:12.563226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af075bca-b3f4-4a45-8a13-ddeddd2898af None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:12.563992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af075bca-b3f4-4a45-8a13-ddeddd2898af None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:12.564733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af075bca-b3f4-4a45-8a13-ddeddd2898af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:12.597754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af075bca-b3f4-4a45-8a13-ddeddd2898af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:12.601115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af075bca-b3f4-4a45-8a13-ddeddd2898af None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-af075bca-b3f4-4a45-8a13-ddeddd2898af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:12.634005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:12.635755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:12.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:12.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:12.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:12.637443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:12.641426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 418/837] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:12.648219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7f178-5e75-44c4-989f-f99498f89d18 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:12.730995 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8c27c-38de-43f1-8696-45dd905eb964 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:12.731286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8c27c-38de-43f1-8696-45dd905eb964 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:12.731641 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8c27c-38de-43f1-8696-45dd905eb964 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:12.733250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 419/838] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:12.742130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d6f24-b4fb-4a64-9ebb-ec40833aa54d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:12.742578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d6f24-b4fb-4a64-9ebb-ec40833aa54d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:12.743150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d6f24-b4fb-4a64-9ebb-ec40833aa54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:12.772117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2f3d6f24-b4fb-4a64-9ebb-ec40833aa54d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:12.774403 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2f3d6f24-b4fb-4a64-9ebb-ec40833aa54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:12.826213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7f178-5e75-44c4-989f-f99498f89d18 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:12.884180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b29e1-4af0-4556-ae63-e6ec6b864ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:12.885117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b29e1-4af0-4556-ae63-e6ec6b864ea8 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:12.885582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b29e1-4af0-4556-ae63-e6ec6b864ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:12.886009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b29e1-4af0-4556-ae63-e6ec6b864ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:12.886949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b29e1-4af0-4556-ae63-e6ec6b864ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:12.887565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b29e1-4af0-4556-ae63-e6ec6b864ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:12.890507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b29e1-4af0-4556-ae63-e6ec6b864ea8 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:12.902827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 420/839] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:12.909591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 420/840] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:12.919966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d6f24-b4fb-4a64-9ebb-ec40833aa54d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:12.936531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b29e1-4af0-4556-ae63-e6ec6b864ea8 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:13.098918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d6f24-b4fb-4a64-9ebb-ec40833aa54d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:13.124508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b29e1-4af0-4556-ae63-e6ec6b864ea8 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:13.156833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c91aa8-d7e2-4c84-b695-453983976d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:13.158391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c91aa8-d7e2-4c84-b695-453983976d8a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:13.158974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c91aa8-d7e2-4c84-b695-453983976d8a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:13.159510 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c91aa8-d7e2-4c84-b695-453983976d8a None admin] PATH_INFO: `/v3/users/swift` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:13.171966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c91aa8-d7e2-4c84-b695-453983976d8a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:13.174529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c91aa8-d7e2-4c84-b695-453983976d8a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:13.186784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309a641-b61c-4f81-984a-df91a451b69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:13.188133 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309a641-b61c-4f81-984a-df91a451b69d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:13.188608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309a641-b61c-4f81-984a-df91a451b69d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:13.189061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309a641-b61c-4f81-984a-df91a451b69d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-22c91aa8-d7e2-4c84-b695-453983976d8a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:13.204168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:13.205430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:13.206681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:13.206681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:13.206681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:13.206681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:13.206681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:13.206681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:13.206681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:13.206681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:13.206681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:42:13.206681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:13.213232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309a641-b61c-4f81-984a-df91a451b69d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:13.213493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 421/841] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:42:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:13.218581 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c91aa8-d7e2-4c84-b695-453983976d8a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-f309a641-b61c-4f81-984a-df91a451b69d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:13.223965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:13.225232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:13.226814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 421/842] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:42:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:13.235772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309a641-b61c-4f81-984a-df91a451b69d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:13.373608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c91aa8-d7e2-4c84-b695-453983976d8a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:13.403777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309a641-b61c-4f81-984a-df91a451b69d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:13.424664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b431f-16d1-4ba2-827b-e5280562f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:13.426088 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b431f-16d1-4ba2-827b-e5280562f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:13.426547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b431f-16d1-4ba2-827b-e5280562f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:13.427163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b431f-16d1-4ba2-827b-e5280562f6d3 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:13.428523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b431f-16d1-4ba2-827b-e5280562f6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:13.428837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b431f-16d1-4ba2-827b-e5280562f6d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:13.430392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b431f-16d1-4ba2-827b-e5280562f6d3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:13.457399 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b8542-d032-4df7-867e-71a40c275998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:13.458279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b8542-d032-4df7-867e-71a40c275998 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:13.458661 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b8542-d032-4df7-867e-71a40c275998 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:13.459045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b8542-d032-4df7-867e-71a40c275998 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:13.459885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b8542-d032-4df7-867e-71a40c275998 None admin] RBAC: Adding query filter params (name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:13.460262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b8542-d032-4df7-867e-71a40c275998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:13.461747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b8542-d032-4df7-867e-71a40c275998 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:13.470934 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 422/843] 10.222.0.79 () {56 vars in 1135 bytes} [Fri May 13 13:42:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:13.476433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b431f-16d1-4ba2-827b-e5280562f6d3 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:13.479096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 422/844] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:42:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:13.486479 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b8542-d032-4df7-867e-71a40c275998 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:13.647132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b431f-16d1-4ba2-827b-e5280562f6d3 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:13.647718 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b8542-d032-4df7-867e-71a40c275998 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:13.705236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace98fcd-ed30-4d0f-b90a-412458821e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:13.706190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace98fcd-ed30-4d0f-b90a-412458821e6c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:13.706325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace98fcd-ed30-4d0f-b90a-412458821e6c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:13.706601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace98fcd-ed30-4d0f-b90a-412458821e6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:13.711613 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f8521-5a0d-4e9c-911c-0c642ca87e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:13.713161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f8521-5a0d-4e9c-911c-0c642ca87e2b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:13.713667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f8521-5a0d-4e9c-911c-0c642ca87e2b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:13.714157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f8521-5a0d-4e9c-911c-0c642ca87e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:13.726934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f8521-5a0d-4e9c-911c-0c642ca87e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:13.729572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f8521-5a0d-4e9c-911c-0c642ca87e2b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-849f8521-5a0d-4e9c-911c-0c642ca87e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:13.754800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:13.755753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:13.756657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:13.756657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:13.756657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:13.756657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:13.759207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 423/845] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:13.765229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f8521-5a0d-4e9c-911c-0c642ca87e2b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:13.790461 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace98fcd-ed30-4d0f-b90a-412458821e6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:13.793303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace98fcd-ed30-4d0f-b90a-412458821e6c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:13.889870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f8521-5a0d-4e9c-911c-0c642ca87e2b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:13.914825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 423/846] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:42:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:42:13.916602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b5431-591a-40dd-b44e-1820976978a4 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:13.917057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b5431-591a-40dd-b44e-1820976978a4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:13.917335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b5431-591a-40dd-b44e-1820976978a4 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:13.922775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 424/847] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:13.922775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b5431-591a-40dd-b44e-1820976978a4 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:13.940701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef54eb-dbc2-402a-aadd-57ff1819e71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:13.941374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef54eb-dbc2-402a-aadd-57ff1819e71f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:13.941651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef54eb-dbc2-402a-aadd-57ff1819e71f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:13.941925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef54eb-dbc2-402a-aadd-57ff1819e71f None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:13.942834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef54eb-dbc2-402a-aadd-57ff1819e71f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:13.943142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef54eb-dbc2-402a-aadd-57ff1819e71f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:13.944555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef54eb-dbc2-402a-aadd-57ff1819e71f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:13.958782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 424/848] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:13.960099 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b16a-2059-4730-82ce-84bf49965b7d None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:13.960378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b16a-2059-4730-82ce-84bf49965b7d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:13.960649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6b16a-2059-4730-82ce-84bf49965b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:13.984698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-21e6b16a-2059-4730-82ce-84bf49965b7d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:13.986004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-21e6b16a-2059-4730-82ce-84bf49965b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:14.063254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b5431-591a-40dd-b44e-1820976978a4 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:14.085992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 425/849] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:14.085992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6b16a-2059-4730-82ce-84bf49965b7d None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:14.130900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf2f89-4c0c-4179-9ddc-348cbf6c97a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:14.133804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf2f89-4c0c-4179-9ddc-348cbf6c97a3 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:14.133804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf2f89-4c0c-4179-9ddc-348cbf6c97a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:14.133804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf2f89-4c0c-4179-9ddc-348cbf6c97a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:14.150157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf2f89-4c0c-4179-9ddc-348cbf6c97a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:14.152795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf2f89-4c0c-4179-9ddc-348cbf6c97a3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-f7bf2f89-4c0c-4179-9ddc-348cbf6c97a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:14.182652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:14.184507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:14.187171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:14.187171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:14.187171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:14.187171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:14.194456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 425/850] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:14.194456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf2f89-4c0c-4179-9ddc-348cbf6c97a3 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:14.243251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6b16a-2059-4730-82ce-84bf49965b7d None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:14.288513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6092d7-4680-4fdc-88dd-6d658c1109c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:14.289624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6092d7-4680-4fdc-88dd-6d658c1109c8 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:14.289915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6092d7-4680-4fdc-88dd-6d658c1109c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:14.290296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6092d7-4680-4fdc-88dd-6d658c1109c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:14.298766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6092d7-4680-4fdc-88dd-6d658c1109c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:14.301069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6092d7-4680-4fdc-88dd-6d658c1109c8 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-0a6092d7-4680-4fdc-88dd-6d658c1109c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:14.318367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:14.319911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 426/851] 10.222.0.79 () {56 vars in 1117 bytes} [Fri May 13 13:42:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:14.322449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6092d7-4680-4fdc-88dd-6d658c1109c8 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:14.365177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf2f89-4c0c-4179-9ddc-348cbf6c97a3 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:14.415118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dce77-9096-44e7-b253-01f9b0452a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:14.416692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dce77-9096-44e7-b253-01f9b0452a53 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:14.417188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dce77-9096-44e7-b253-01f9b0452a53 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:14.417612 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dce77-9096-44e7-b253-01f9b0452a53 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:14.418844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dce77-9096-44e7-b253-01f9b0452a53 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:14.421424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dce77-9096-44e7-b253-01f9b0452a53 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-620dce77-9096-44e7-b253-01f9b0452a53 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:14.443225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:14.444522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:14.446230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 426/852] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:42:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:14.448475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dce77-9096-44e7-b253-01f9b0452a53 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:14.480124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6092d7-4680-4fdc-88dd-6d658c1109c8 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:14.530949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a9208-e524-40ac-845a-18e950715640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:14.532054 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a9208-e524-40ac-845a-18e950715640 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:14.532354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a9208-e524-40ac-845a-18e950715640 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:14.532666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a9208-e524-40ac-845a-18e950715640 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:14.533540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a9208-e524-40ac-845a-18e950715640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:14.533853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a9208-e524-40ac-845a-18e950715640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:14.535987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a9208-e524-40ac-845a-18e950715640 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:14.549172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 427/853] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:14.550523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a9208-e524-40ac-845a-18e950715640 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:14.607668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dce77-9096-44e7-b253-01f9b0452a53 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:14.665311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b7314-1f3a-402c-b87d-8fd917991a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:14.666359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b7314-1f3a-402c-b87d-8fd917991a7f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:14.666819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b7314-1f3a-402c-b87d-8fd917991a7f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:14.667252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b7314-1f3a-402c-b87d-8fd917991a7f None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:14.668714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b7314-1f3a-402c-b87d-8fd917991a7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:14.669204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b7314-1f3a-402c-b87d-8fd917991a7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:14.671258 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b7314-1f3a-402c-b87d-8fd917991a7f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:14.701475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 427/854] 10.222.0.79 () {56 vars in 1149 bytes} [Fri May 13 13:42:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:14.702558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b7314-1f3a-402c-b87d-8fd917991a7f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:14.726562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a9208-e524-40ac-845a-18e950715640 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:14.787144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf70d3a-28ab-447f-9eb1-898837bd7535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:14.788157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf70d3a-28ab-447f-9eb1-898837bd7535 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:14.788652 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf70d3a-28ab-447f-9eb1-898837bd7535 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:14.789144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf70d3a-28ab-447f-9eb1-898837bd7535 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:14.790483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf70d3a-28ab-447f-9eb1-898837bd7535 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:14.791263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf70d3a-28ab-447f-9eb1-898837bd7535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:14.793651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf70d3a-28ab-447f-9eb1-898837bd7535 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:14.802267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 428/855] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:42:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:14.804513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980884e-f221-4dfb-a436-dddd4c7f6ae1 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:14.805014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980884e-f221-4dfb-a436-dddd4c7f6ae1 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:14.805496 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980884e-f221-4dfb-a436-dddd4c7f6ae1 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:14.807840 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 429/856] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:14.809106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980884e-f221-4dfb-a436-dddd4c7f6ae1 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:14.838394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b7314-1f3a-402c-b87d-8fd917991a7f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:14.890191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1056747-82d2-486f-a893-ad047d716876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:14.891509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1056747-82d2-486f-a893-ad047d716876 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:14.891984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1056747-82d2-486f-a893-ad047d716876 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:14.892446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1056747-82d2-486f-a893-ad047d716876 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:14.905067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1056747-82d2-486f-a893-ad047d716876 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:14.907709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1056747-82d2-486f-a893-ad047d716876 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-f1056747-82d2-486f-a893-ad047d716876 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:14.933921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:14.935653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:14.936593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:14.936593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:14.936593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:14.936593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:14.936593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:14.936593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:14.936593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:14.936593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:14.936593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 428/857] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:14.937950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1056747-82d2-486f-a893-ad047d716876 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:14.968137 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980884e-f221-4dfb-a436-dddd4c7f6ae1 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:15.006448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cbbae-28bc-419d-bd11-895aaa7eb128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:15.007195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbbae-28bc-419d-bd11-895aaa7eb128 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:15.007485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbbae-28bc-419d-bd11-895aaa7eb128 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:15.007760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbbae-28bc-419d-bd11-895aaa7eb128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:15.090310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1056747-82d2-486f-a893-ad047d716876 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:15.099124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cbbae-28bc-419d-bd11-895aaa7eb128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:15.100863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cbbae-28bc-419d-bd11-895aaa7eb128 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:15.135858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc149f-4898-4470-8e95-96a8b378a2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:15.136536 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc149f-4898-4470-8e95-96a8b378a2bc None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:15.136817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc149f-4898-4470-8e95-96a8b378a2bc None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:15.137097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc149f-4898-4470-8e95-96a8b378a2bc None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:15.137809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc149f-4898-4470-8e95-96a8b378a2bc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:15.138117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc149f-4898-4470-8e95-96a8b378a2bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:15.139552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc149f-4898-4470-8e95-96a8b378a2bc None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:15.145795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 429/858] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:42:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:15.149702 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde5350-15da-44c0-a319-7be71684dbbe None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:15.150137 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde5350-15da-44c0-a319-7be71684dbbe None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:15.150586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde5350-15da-44c0-a319-7be71684dbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:15.179396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-bfde5350-15da-44c0-a319-7be71684dbbe None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:15.181321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-bfde5350-15da-44c0-a319-7be71684dbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:15.242420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 430/859] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:42:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:42:15.242420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cbbae-28bc-419d-bd11-895aaa7eb128 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:15.306973 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 430/860] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:15.308022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde5350-15da-44c0-a319-7be71684dbbe None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:15.378972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cbbae-28bc-419d-bd11-895aaa7eb128 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:15.430001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e34dd-3d4b-4bc7-b708-30938bc628bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:15.431009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e34dd-3d4b-4bc7-b708-30938bc628bd None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:15.431443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e34dd-3d4b-4bc7-b708-30938bc628bd None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:15.431882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e34dd-3d4b-4bc7-b708-30938bc628bd None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:15.433257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e34dd-3d4b-4bc7-b708-30938bc628bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:15.433748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e34dd-3d4b-4bc7-b708-30938bc628bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:15.436086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e34dd-3d4b-4bc7-b708-30938bc628bd None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:15.464809 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 431/861] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:15.464809 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e34dd-3d4b-4bc7-b708-30938bc628bd None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:15.465191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde5350-15da-44c0-a319-7be71684dbbe None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:15.520984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66236288-0b7d-40d3-b9d1-377bbf8b7e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:15.522395 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66236288-0b7d-40d3-b9d1-377bbf8b7e72 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:15.522936 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66236288-0b7d-40d3-b9d1-377bbf8b7e72 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:15.523437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66236288-0b7d-40d3-b9d1-377bbf8b7e72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:15.541599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66236288-0b7d-40d3-b9d1-377bbf8b7e72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:15.543755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66236288-0b7d-40d3-b9d1-377bbf8b7e72 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-66236288-0b7d-40d3-b9d1-377bbf8b7e72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:15.564695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:15.566328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:15.567913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:15.567913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:15.567913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:15.567913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:15.570968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 431/862] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:15.573001 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66236288-0b7d-40d3-b9d1-377bbf8b7e72 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:15.654583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e34dd-3d4b-4bc7-b708-30938bc628bd None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:15.721766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b9088-72c4-4be4-9996-79e8bbd8245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:15.722595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b9088-72c4-4be4-9996-79e8bbd8245c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:15.722898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b9088-72c4-4be4-9996-79e8bbd8245c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:15.723744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b9088-72c4-4be4-9996-79e8bbd8245c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:15.724582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b9088-72c4-4be4-9996-79e8bbd8245c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:15.726016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b9088-72c4-4be4-9996-79e8bbd8245c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:15.726588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66236288-0b7d-40d3-b9d1-377bbf8b7e72 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-a42b9088-72c4-4be4-9996-79e8bbd8245c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:15.748895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:15.751191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:42:15.753139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:15.759140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 432/863] 10.222.0.79 () {56 vars in 1132 bytes} [Fri May 13 13:42:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:15.760385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b9088-72c4-4be4-9996-79e8bbd8245c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:15.780514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109d282-6992-4593-acd6-3d980c5adec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:15.780978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109d282-6992-4593-acd6-3d980c5adec0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:15.781474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109d282-6992-4593-acd6-3d980c5adec0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:15.781584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109d282-6992-4593-acd6-3d980c5adec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:15.782461 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109d282-6992-4593-acd6-3d980c5adec0 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=4139cfe906bd4c7d94226ab589577627) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:15.782808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109d282-6992-4593-acd6-3d980c5adec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:15.784892 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109d282-6992-4593-acd6-3d980c5adec0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:15.791409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 432/864] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:42:15 2022] GET /identity/v3/role_assignments?user.id=4139cfe906bd4c7d94226ab589577627&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:15.792673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109d282-6992-4593-acd6-3d980c5adec0 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:15.912216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b9088-72c4-4be4-9996-79e8bbd8245c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:15.977805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9142261-3819-43f2-b6d8-7608f94b7f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:15.979555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9142261-3819-43f2-b6d8-7608f94b7f2c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:15.980354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9142261-3819-43f2-b6d8-7608f94b7f2c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:15.980819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109d282-6992-4593-acd6-3d980c5adec0 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:15.981428 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9142261-3819-43f2-b6d8-7608f94b7f2c None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:15.983238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9142261-3819-43f2-b6d8-7608f94b7f2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:15.984452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9142261-3819-43f2-b6d8-7608f94b7f2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:15.990499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9142261-3819-43f2-b6d8-7608f94b7f2c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:16.021643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 433/865] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:16.028933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9142261-3819-43f2-b6d8-7608f94b7f2c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:16.037541 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283316b-0b20-462f-a563-b3a6931349fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:16.038477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283316b-0b20-462f-a563-b3a6931349fc None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:16.038650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283316b-0b20-462f-a563-b3a6931349fc None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:16.042653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283316b-0b20-462f-a563-b3a6931349fc None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:16.042653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283316b-0b20-462f-a563-b3a6931349fc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:16.042653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283316b-0b20-462f-a563-b3a6931349fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:16.042653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283316b-0b20-462f-a563-b3a6931349fc None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:16.057011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 433/866] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:16.059990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283316b-0b20-462f-a563-b3a6931349fc None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:16.199230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283316b-0b20-462f-a563-b3a6931349fc None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:16.201107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9142261-3819-43f2-b6d8-7608f94b7f2c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:16.237520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597e5b-0e24-4fcf-8ac6-3669d8725876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:16.238198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597e5b-0e24-4fcf-8ac6-3669d8725876 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:16.238485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597e5b-0e24-4fcf-8ac6-3669d8725876 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:16.238787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597e5b-0e24-4fcf-8ac6-3669d8725876 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:16.239498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597e5b-0e24-4fcf-8ac6-3669d8725876 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:16.239812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597e5b-0e24-4fcf-8ac6-3669d8725876 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:16.239929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77e713-2a2d-4635-b9b5-3529bf9cc63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:16.240688 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77e713-2a2d-4635-b9b5-3529bf9cc63d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:16.240968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77e713-2a2d-4635-b9b5-3529bf9cc63d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:16.241199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597e5b-0e24-4fcf-8ac6-3669d8725876 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:16.241301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77e713-2a2d-4635-b9b5-3529bf9cc63d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:16.246574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 434/867] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:16.249601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77e713-2a2d-4635-b9b5-3529bf9cc63d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:16.258771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77e713-2a2d-4635-b9b5-3529bf9cc63d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-0b77e713-2a2d-4635-b9b5-3529bf9cc63d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:16.267846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:16.269418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:16.270700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:16.270700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:16.270700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:16.270700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:16.270700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:16.270700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:16.270700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:16.270700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:16.274803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 434/868] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:16.276276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597e5b-0e24-4fcf-8ac6-3669d8725876 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:16.419192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597e5b-0e24-4fcf-8ac6-3669d8725876 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:16.480875 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1468e8c-71dc-4f38-be00-a3e6cf2fd486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:16.481915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1468e8c-71dc-4f38-be00-a3e6cf2fd486 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:16.482350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1468e8c-71dc-4f38-be00-a3e6cf2fd486 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:16.482885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1468e8c-71dc-4f38-be00-a3e6cf2fd486 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:16.484151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1468e8c-71dc-4f38-be00-a3e6cf2fd486 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:16.484560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1468e8c-71dc-4f38-be00-a3e6cf2fd486 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:16.486281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1468e8c-71dc-4f38-be00-a3e6cf2fd486 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:16.501169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 435/869] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:16.510494 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77e713-2a2d-4635-b9b5-3529bf9cc63d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:16.684944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77e713-2a2d-4635-b9b5-3529bf9cc63d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:16.741202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da50e10-1257-4cdf-9905-56853b3fa0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:16.742344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50e10-1257-4cdf-9905-56853b3fa0e6 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:16.742879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50e10-1257-4cdf-9905-56853b3fa0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:16.743471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da50e10-1257-4cdf-9905-56853b3fa0e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:16.744677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da50e10-1257-4cdf-9905-56853b3fa0e6 None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=598c323e8ed5455cbe4a568b360d8eab) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:16.745239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da50e10-1257-4cdf-9905-56853b3fa0e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:16.746931 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da50e10-1257-4cdf-9905-56853b3fa0e6 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:16.758700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 435/870] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:42:16 2022] GET /identity/v3/role_assignments?user.id=598c323e8ed5455cbe4a568b360d8eab&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:17.825710 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feaf50-1573-4fec-a985-bb6a359c76f0 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:17.826497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feaf50-1573-4fec-a985-bb6a359c76f0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:17.826497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feaf50-1573-4fec-a985-bb6a359c76f0 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:17.827930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 436/871] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:17.834381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9f709-2069-460a-8d1b-83dd69d6a2c5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:17.834939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9f709-2069-460a-8d1b-83dd69d6a2c5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:17.834939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9f709-2069-460a-8d1b-83dd69d6a2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:17.853559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-21b9f709-2069-460a-8d1b-83dd69d6a2c5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:17.855135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-21b9f709-2069-460a-8d1b-83dd69d6a2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:17.925766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 436/872] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:17.948363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5feaf50-1573-4fec-a985-bb6a359c76f0 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:18.059964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5feaf50-1573-4fec-a985-bb6a359c76f0 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:18.097593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc29d6-0b70-49b1-a6c2-c4c5d5ddeb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:18.098273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc29d6-0b70-49b1-a6c2-c4c5d5ddeb68 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:18.098559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc29d6-0b70-49b1-a6c2-c4c5d5ddeb68 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:18.098854 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc29d6-0b70-49b1-a6c2-c4c5d5ddeb68 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:18.099589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc29d6-0b70-49b1-a6c2-c4c5d5ddeb68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:18.101621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc29d6-0b70-49b1-a6c2-c4c5d5ddeb68 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:18.119247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 437/873] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:42:17 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:42:18.573934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9da031-be9e-4609-8c0b-4a879abae0c0 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:18.574331 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9da031-be9e-4609-8c0b-4a879abae0c0 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:18.574810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9da031-be9e-4609-8c0b-4a879abae0c0 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:18.576431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 437/874] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:18.582910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3350b-da8d-464f-a40a-157d9a8f6213 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:18.583287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3350b-da8d-464f-a40a-157d9a8f6213 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:18.583573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3350b-da8d-464f-a40a-157d9a8f6213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:18.606675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-25a3350b-da8d-464f-a40a-157d9a8f6213 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:18.607960 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-25a3350b-da8d-464f-a40a-157d9a8f6213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:18.710281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 438/875] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:18.734824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9da031-be9e-4609-8c0b-4a879abae0c0 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:18.750168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b3268-f8b8-42a9-925e-f920457cf22b None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:18.750507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b3268-f8b8-42a9-925e-f920457cf22b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:18.750755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b3268-f8b8-42a9-925e-f920457cf22b None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:18.752544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 439/876] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:18.758159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a168b8e-3be8-4664-a839-57488c081d30 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:18.758455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a168b8e-3be8-4664-a839-57488c081d30 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:18.758739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a168b8e-3be8-4664-a839-57488c081d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:18.777419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1a168b8e-3be8-4664-a839-57488c081d30 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:18.778645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1a168b8e-3be8-4664-a839-57488c081d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:18.859076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 440/877] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:18.871758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9da031-be9e-4609-8c0b-4a879abae0c0 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:18.877580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a168b8e-3be8-4664-a839-57488c081d30 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:18.909066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85b001-6edd-404f-8ef5-884f894d43f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:18.909887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85b001-6edd-404f-8ef5-884f894d43f8 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:18.910167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85b001-6edd-404f-8ef5-884f894d43f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:18.910440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85b001-6edd-404f-8ef5-884f894d43f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:18.911137 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85b001-6edd-404f-8ef5-884f894d43f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:18.914764 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85b001-6edd-404f-8ef5-884f894d43f8 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-6e85b001-6edd-404f-8ef5-884f894d43f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:18.928733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:18.930114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:18.930990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 438/878] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:18.936484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85b001-6edd-404f-8ef5-884f894d43f8 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:19.010536 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a168b8e-3be8-4664-a839-57488c081d30 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:19.057768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6dfd4-619f-41cb-90e0-34469adbb231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:19.058457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6dfd4-619f-41cb-90e0-34469adbb231 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:19.058759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6dfd4-619f-41cb-90e0-34469adbb231 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:19.059042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6dfd4-619f-41cb-90e0-34469adbb231 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:19.059791 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6dfd4-619f-41cb-90e0-34469adbb231 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:19.061224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6dfd4-619f-41cb-90e0-34469adbb231 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:19.077595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85b001-6edd-404f-8ef5-884f894d43f8 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:19.084482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 441/879] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:42:18 2022] POST /identity/v3/services => generated 269 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:42:19.132669 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ada1ef-9a42-4072-a1af-24c03157b990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:19.133761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ada1ef-9a42-4072-a1af-24c03157b990 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:19.134268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ada1ef-9a42-4072-a1af-24c03157b990 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:19.134770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ada1ef-9a42-4072-a1af-24c03157b990 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:19.135733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ada1ef-9a42-4072-a1af-24c03157b990 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:19.136250 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ada1ef-9a42-4072-a1af-24c03157b990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:19.137997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ada1ef-9a42-4072-a1af-24c03157b990 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:19.145144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 439/880] 10.222.0.79 () {56 vars in 1122 bytes} [Fri May 13 13:42:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:19.151711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6dfd4-619f-41cb-90e0-34469adbb231 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:19.280776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6dfd4-619f-41cb-90e0-34469adbb231 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:19.320780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e8333-91b9-416e-aa89-fe744b30376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:19.321358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e8333-91b9-416e-aa89-fe744b30376c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:19.321611 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e8333-91b9-416e-aa89-fe744b30376c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:19.321881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e8333-91b9-416e-aa89-fe744b30376c None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:19.322624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e8333-91b9-416e-aa89-fe744b30376c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:19.323029 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e8333-91b9-416e-aa89-fe744b30376c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:19.324653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e8333-91b9-416e-aa89-fe744b30376c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:19.332388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 442/881] 10.222.0.79 () {56 vars in 1122 bytes} [Fri May 13 13:42:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:19.469239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df8a62-711a-4db4-a42b-9703650ca131 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:19.469741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df8a62-711a-4db4-a42b-9703650ca131 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:19.470090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df8a62-711a-4db4-a42b-9703650ca131 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:19.472712 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 440/882] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:19.480634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b797-1042-4f08-ba74-dc3ac55e9d92 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:19.480951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b797-1042-4f08-ba74-dc3ac55e9d92 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:19.481343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2b797-1042-4f08-ba74-dc3ac55e9d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:19.505010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-12b2b797-1042-4f08-ba74-dc3ac55e9d92 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:19.506274 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-12b2b797-1042-4f08-ba74-dc3ac55e9d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:19.600112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 443/883] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:19.623821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df8a62-711a-4db4-a42b-9703650ca131 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:19.751672 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df8a62-711a-4db4-a42b-9703650ca131 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:19.793904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78809182-8389-44fd-89c9-f020bbd45a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:19.795183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78809182-8389-44fd-89c9-f020bbd45a9f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:19.795679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78809182-8389-44fd-89c9-f020bbd45a9f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:19.796236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78809182-8389-44fd-89c9-f020bbd45a9f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:19.808552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78809182-8389-44fd-89c9-f020bbd45a9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-78809182-8389-44fd-89c9-f020bbd45a9f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:19.817973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:19.818970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:19.819841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:19.824906 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 441/884] 10.222.0.79 () {56 vars in 1120 bytes} [Fri May 13 13:42:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:19.834751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2b797-1042-4f08-ba74-dc3ac55e9d92 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:19.985218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2b797-1042-4f08-ba74-dc3ac55e9d92 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:20.034815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ecc83-edd3-48b1-9899-5b0e49489647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:20.035796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ecc83-edd3-48b1-9899-5b0e49489647 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:20.036249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ecc83-edd3-48b1-9899-5b0e49489647 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:20.036695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ecc83-edd3-48b1-9899-5b0e49489647 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:20.037991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ecc83-edd3-48b1-9899-5b0e49489647 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:20.038492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ecc83-edd3-48b1-9899-5b0e49489647 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:20.040878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ecc83-edd3-48b1-9899-5b0e49489647 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:20.056134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 444/885] 10.222.0.79 () {56 vars in 1115 bytes} [Fri May 13 13:42:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:20.066127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78809182-8389-44fd-89c9-f020bbd45a9f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:20.196214 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78809182-8389-44fd-89c9-f020bbd45a9f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:20.248102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeee69-f9ca-40c0-a7a1-294d8bf52684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:20.249402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeee69-f9ca-40c0-a7a1-294d8bf52684 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:20.249872 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeee69-f9ca-40c0-a7a1-294d8bf52684 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:20.250413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeee69-f9ca-40c0-a7a1-294d8bf52684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:20.269397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caeee69-f9ca-40c0-a7a1-294d8bf52684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:20.272094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caeee69-f9ca-40c0-a7a1-294d8bf52684 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-2caeee69-f9ca-40c0-a7a1-294d8bf52684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:20.305441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:20.307659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:20.309789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:20.309789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:20.309789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:20.309789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:20.314851 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 442/886] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:20.320164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ecc83-edd3-48b1-9899-5b0e49489647 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:20.443591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ecc83-edd3-48b1-9899-5b0e49489647 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:20.481488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41560a-e03b-48b3-a6c5-8bd1233918df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:20.482570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41560a-e03b-48b3-a6c5-8bd1233918df None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:20.483157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41560a-e03b-48b3-a6c5-8bd1233918df None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:20.483695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41560a-e03b-48b3-a6c5-8bd1233918df None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:20.484640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41560a-e03b-48b3-a6c5-8bd1233918df None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:20.485246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41560a-e03b-48b3-a6c5-8bd1233918df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:20.487126 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41560a-e03b-48b3-a6c5-8bd1233918df None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:20.502217 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 445/887] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:20.511400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeee69-f9ca-40c0-a7a1-294d8bf52684 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:20.640136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeee69-f9ca-40c0-a7a1-294d8bf52684 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:20.679388 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dcb9a-727f-4a49-8029-e069b656b918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:20.680231 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dcb9a-727f-4a49-8029-e069b656b918 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:20.680508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dcb9a-727f-4a49-8029-e069b656b918 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:20.680790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dcb9a-727f-4a49-8029-e069b656b918 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:20.689611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dcb9a-727f-4a49-8029-e069b656b918 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:20.691145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dcb9a-727f-4a49-8029-e069b656b918 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-a43dcb9a-727f-4a49-8029-e069b656b918 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:20.707450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:20.708459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:20.709400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:20.709400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:20.709400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:20.709400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:20.709400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:20.709400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:20.709400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:20.709400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:20.709400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:42:20.709400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:20.713360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 443/888] 10.222.0.79 () {56 vars in 1117 bytes} [Fri May 13 13:42:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:20.714733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41560a-e03b-48b3-a6c5-8bd1233918df None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:20.825827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41560a-e03b-48b3-a6c5-8bd1233918df None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:20.867053 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b223c-15d0-43df-bba7-91c90140a6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:20.868134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b223c-15d0-43df-bba7-91c90140a6ab None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:20.868623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b223c-15d0-43df-bba7-91c90140a6ab None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:20.869099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b223c-15d0-43df-bba7-91c90140a6ab None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:20.870144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b223c-15d0-43df-bba7-91c90140a6ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:20.870679 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b223c-15d0-43df-bba7-91c90140a6ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:20.872416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b223c-15d0-43df-bba7-91c90140a6ab None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:20.883382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018520a3-3de2-4092-98c8-64e76f2a6238 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:20.883862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018520a3-3de2-4092-98c8-64e76f2a6238 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:20.884349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018520a3-3de2-4092-98c8-64e76f2a6238 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:20.891126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 444/889] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:20.896934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81562d-dcdc-46bd-b1fd-97f1a75cc957 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:20.897384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81562d-dcdc-46bd-b1fd-97f1a75cc957 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:20.897837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81562d-dcdc-46bd-b1fd-97f1a75cc957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:20.899827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 446/890] 10.222.0.79 () {56 vars in 1149 bytes} [Fri May 13 13:42:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:20.906752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b223c-15d0-43df-bba7-91c90140a6ab None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:20.931483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-ee81562d-dcdc-46bd-b1fd-97f1a75cc957 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:20.933455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ee81562d-dcdc-46bd-b1fd-97f1a75cc957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:21.012010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 445/891] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:21.051833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81562d-dcdc-46bd-b1fd-97f1a75cc957 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:21.077264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b223c-15d0-43df-bba7-91c90140a6ab None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:21.144687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0895d8-ceb0-4583-a4dc-79089fefdae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:21.146037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0895d8-ceb0-4583-a4dc-79089fefdae6 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:21.146320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0895d8-ceb0-4583-a4dc-79089fefdae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:21.146595 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0895d8-ceb0-4583-a4dc-79089fefdae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:21.173626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81562d-dcdc-46bd-b1fd-97f1a75cc957 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:21.229988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b0aa0-9dad-45ad-8f2b-67c50d638d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:21.232347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b0aa0-9dad-45ad-8f2b-67c50d638d08 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:21.232984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b0aa0-9dad-45ad-8f2b-67c50d638d08 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:21.233963 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b0aa0-9dad-45ad-8f2b-67c50d638d08 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:21.235584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b0aa0-9dad-45ad-8f2b-67c50d638d08 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:21.239795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b0aa0-9dad-45ad-8f2b-67c50d638d08 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:21.247804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0895d8-ceb0-4583-a4dc-79089fefdae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:21.250158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0895d8-ceb0-4583-a4dc-79089fefdae6 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-5b4b0aa0-9dad-45ad-8f2b-67c50d638d08 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:21.259714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:21.260866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:21.261852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 446/892] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:42:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:21.269460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b0aa0-9dad-45ad-8f2b-67c50d638d08 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:21.409334 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 447/893] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:42:20 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:42:21.409516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b0aa0-9dad-45ad-8f2b-67c50d638d08 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:21.415526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0895d8-ceb0-4583-a4dc-79089fefdae6 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:21.457519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b2919-d5bc-4c37-a684-ffb5857fe685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:21.458223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2919-d5bc-4c37-a684-ffb5857fe685 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:21.458829 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2919-d5bc-4c37-a684-ffb5857fe685 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:21.459136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2919-d5bc-4c37-a684-ffb5857fe685 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:21.459903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b2919-d5bc-4c37-a684-ffb5857fe685 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:21.460210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b2919-d5bc-4c37-a684-ffb5857fe685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:21.462614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b2919-d5bc-4c37-a684-ffb5857fe685 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:21.470192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 447/894] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:42:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:21.478856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b2919-d5bc-4c37-a684-ffb5857fe685 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:21.568426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0895d8-ceb0-4583-a4dc-79089fefdae6 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:21.615814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b2919-d5bc-4c37-a684-ffb5857fe685 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:21.637607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcd78a-d726-4b38-9fd6-f7ddf328294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:21.639352 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcd78a-d726-4b38-9fd6-f7ddf328294f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:21.639881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcd78a-d726-4b38-9fd6-f7ddf328294f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:21.640661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcd78a-d726-4b38-9fd6-f7ddf328294f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:21.656610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcd78a-d726-4b38-9fd6-f7ddf328294f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:21.659879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcd78a-d726-4b38-9fd6-f7ddf328294f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:21.661508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e969325-93c0-49e0-a617-b45b5fea4e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:21.662602 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e969325-93c0-49e0-a617-b45b5fea4e0f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:21.663117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e969325-93c0-49e0-a617-b45b5fea4e0f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:21.663586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e969325-93c0-49e0-a617-b45b5fea4e0f None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:21.664860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e969325-93c0-49e0-a617-b45b5fea4e0f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:21.665375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e969325-93c0-49e0-a617-b45b5fea4e0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:21.667943 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e969325-93c0-49e0-a617-b45b5fea4e0f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:21.679104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 448/895] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:42:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:21.682308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e969325-93c0-49e0-a617-b45b5fea4e0f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-dedcd78a-d726-4b38-9fd6-f7ddf328294f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:21.687049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:21.688023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:21.689023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:21.689023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:21.689023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:21.689023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:21.689023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 448/896] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:21.700752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcd78a-d726-4b38-9fd6-f7ddf328294f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:21.821159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e969325-93c0-49e0-a617-b45b5fea4e0f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:21.836446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcd78a-d726-4b38-9fd6-f7ddf328294f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:21.866002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887377d-5748-40a5-9abb-4d3f586b5d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:21.867150 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887377d-5748-40a5-9abb-4d3f586b5d61 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:21.867633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887377d-5748-40a5-9abb-4d3f586b5d61 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:21.868115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887377d-5748-40a5-9abb-4d3f586b5d61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:21.869542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887377d-5748-40a5-9abb-4d3f586b5d61 None admin] RBAC: Adding query filter params (interface=public, service_id=21e973048a1b43918eba6cd92850bffe, region_id=RegionOne) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:21.870088 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887377d-5748-40a5-9abb-4d3f586b5d61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:21.872766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887377d-5748-40a5-9abb-4d3f586b5d61 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:21.873696 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188eda4-05be-4383-92cd-d6c7ab5822c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:21.874370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188eda4-05be-4383-92cd-d6c7ab5822c1 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:21.874653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188eda4-05be-4383-92cd-d6c7ab5822c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:21.874974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188eda4-05be-4383-92cd-d6c7ab5822c1 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:21.875707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188eda4-05be-4383-92cd-d6c7ab5822c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:21.876017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188eda4-05be-4383-92cd-d6c7ab5822c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:21.877405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188eda4-05be-4383-92cd-d6c7ab5822c1 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:21.892085 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 449/897] 10.222.0.79 () {56 vars in 1251 bytes} [Fri May 13 13:42:21 2022] GET /identity/v3/endpoints?service_id=21e973048a1b43918eba6cd92850bffe&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:21.892085 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f538ba-f12b-49f5-ba9b-0f5617eb6e19 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:21.892085 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f538ba-f12b-49f5-ba9b-0f5617eb6e19 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:21.892085 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f538ba-f12b-49f5-ba9b-0f5617eb6e19 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:21.892085 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 450/898] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:21.893064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f538ba-f12b-49f5-ba9b-0f5617eb6e19 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:21.907502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 449/899] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:21.907502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8d668-fb89-483c-82fe-9231dda87a5f None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:21.907502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8d668-fb89-483c-82fe-9231dda87a5f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:21.907502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8d668-fb89-483c-82fe-9231dda87a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:21.927833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6ad8d668-fb89-483c-82fe-9231dda87a5f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:21.929242 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6ad8d668-fb89-483c-82fe-9231dda87a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:22.026800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 450/900] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:22.026800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8d668-fb89-483c-82fe-9231dda87a5f None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:22.042685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f538ba-f12b-49f5-ba9b-0f5617eb6e19 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:22.090885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dfd65-63a8-425b-be50-50aa2a893430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:22.091885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dfd65-63a8-425b-be50-50aa2a893430 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:22.092290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dfd65-63a8-425b-be50-50aa2a893430 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:22.092730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dfd65-63a8-425b-be50-50aa2a893430 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:22.093941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2dfd65-63a8-425b-be50-50aa2a893430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:22.096527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2dfd65-63a8-425b-be50-50aa2a893430 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:22.110756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 451/901] 10.222.0.79 () {56 vars in 1087 bytes} [Fri May 13 13:42:21 2022] GET /identity/v3/services => generated 1268 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:42:22.110756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dfd65-63a8-425b-be50-50aa2a893430 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:22.212743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8d668-fb89-483c-82fe-9231dda87a5f None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:22.257211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dfd65-63a8-425b-be50-50aa2a893430 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:22.264449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c58604-493a-40c4-90ee-36c85acbb4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:22.264449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c58604-493a-40c4-90ee-36c85acbb4a2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:22.264449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c58604-493a-40c4-90ee-36c85acbb4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:22.265525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c58604-493a-40c4-90ee-36c85acbb4a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:22.273705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c58604-493a-40c4-90ee-36c85acbb4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:22.275300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c58604-493a-40c4-90ee-36c85acbb4a2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-74c58604-493a-40c4-90ee-36c85acbb4a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:22.290619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:22.292338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:22.293994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:22.293994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:22.293994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:22.293994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:22.293994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:22.293994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:22.293994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:22.293994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:22.297333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 451/902] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:22.299089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc19ac-e00c-4b5c-898b-d1f303b8f295 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:22.299927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc19ac-e00c-4b5c-898b-d1f303b8f295 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:22.300254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc19ac-e00c-4b5c-898b-d1f303b8f295 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:22.301899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 452/903] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:22.307834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc19ac-e00c-4b5c-898b-d1f303b8f295 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:22.316487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727457e-78e0-4735-b40c-00f03f691579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:22.317321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727457e-78e0-4735-b40c-00f03f691579 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:22.317578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727457e-78e0-4735-b40c-00f03f691579 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:22.317881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727457e-78e0-4735-b40c-00f03f691579 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:22.318608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727457e-78e0-4735-b40c-00f03f691579 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:22.320093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727457e-78e0-4735-b40c-00f03f691579 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-1727457e-78e0-4735-b40c-00f03f691579 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:22.334030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:22.337322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:22.340228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 452/904] 10.222.0.79 () {56 vars in 1132 bytes} [Fri May 13 13:42:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:22.343555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8080b-a6fd-4905-962f-3803d5828f74 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:22.343555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8080b-a6fd-4905-962f-3803d5828f74 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:22.343555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8080b-a6fd-4905-962f-3803d5828f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:22.357759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-efd8080b-a6fd-4905-962f-3803d5828f74 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:22.359514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-efd8080b-a6fd-4905-962f-3803d5828f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:22.456231 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 453/905] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:22.457777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8080b-a6fd-4905-962f-3803d5828f74 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:22.476197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc19ac-e00c-4b5c-898b-d1f303b8f295 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:22.515760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc450ee0-9634-4b40-a03f-f55976147e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:22.516428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc450ee0-9634-4b40-a03f-f55976147e17 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:22.516711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc450ee0-9634-4b40-a03f-f55976147e17 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:22.516987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc450ee0-9634-4b40-a03f-f55976147e17 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:22.517775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc450ee0-9634-4b40-a03f-f55976147e17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:22.518640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc450ee0-9634-4b40-a03f-f55976147e17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:22.520094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc450ee0-9634-4b40-a03f-f55976147e17 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:22.538410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 453/906] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:22.538410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc450ee0-9634-4b40-a03f-f55976147e17 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:22.597167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8080b-a6fd-4905-962f-3803d5828f74 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:22.632037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d4314-f406-4bf5-b4a8-3965fc9b9a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:22.632717 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4314-f406-4bf5-b4a8-3965fc9b9a41 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:22.633003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4314-f406-4bf5-b4a8-3965fc9b9a41 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:22.633309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4314-f406-4bf5-b4a8-3965fc9b9a41 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:22.634022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d4314-f406-4bf5-b4a8-3965fc9b9a41 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:22.634328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d4314-f406-4bf5-b4a8-3965fc9b9a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:22.635748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d4314-f406-4bf5-b4a8-3965fc9b9a41 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:22.640590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 454/907] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:22.641406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d4314-f406-4bf5-b4a8-3965fc9b9a41 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:22.677485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc450ee0-9634-4b40-a03f-f55976147e17 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:22.724157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df459e69-3dd6-4232-bfe3-5c7cfccb1507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:22.724843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df459e69-3dd6-4232-bfe3-5c7cfccb1507 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:22.725134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df459e69-3dd6-4232-bfe3-5c7cfccb1507 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:22.725403 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df459e69-3dd6-4232-bfe3-5c7cfccb1507 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:22.726323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df459e69-3dd6-4232-bfe3-5c7cfccb1507 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:22.728212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df459e69-3dd6-4232-bfe3-5c7cfccb1507 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:22.744101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 454/908] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:42:22 2022] POST /identity/v3/services => generated 241 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:42:22.744948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df459e69-3dd6-4232-bfe3-5c7cfccb1507 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:22.770112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d4314-f406-4bf5-b4a8-3965fc9b9a41 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:22.821919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf10cb-1b4a-44d0-9848-61dc7f0e550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:22.822622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf10cb-1b4a-44d0-9848-61dc7f0e550e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:22.822937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf10cb-1b4a-44d0-9848-61dc7f0e550e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:22.823217 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf10cb-1b4a-44d0-9848-61dc7f0e550e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:22.824067 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf10cb-1b4a-44d0-9848-61dc7f0e550e None admin] RBAC: Adding query filter params (role.id=f4a90110e2c845408f5ec7685f7ef2d0, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=598c323e8ed5455cbe4a568b360d8eab) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:22.824696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf10cb-1b4a-44d0-9848-61dc7f0e550e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:22.826214 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf10cb-1b4a-44d0-9848-61dc7f0e550e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:22.839310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 455/909] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:42:22 2022] GET /identity/v3/role_assignments?user.id=598c323e8ed5455cbe4a568b360d8eab&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=f4a90110e2c845408f5ec7685f7ef2d0 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:22.927432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df459e69-3dd6-4232-bfe3-5c7cfccb1507 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:22.971433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534bea3-3abc-4717-ac5f-560a7f37dbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:22.972510 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534bea3-3abc-4717-ac5f-560a7f37dbf5 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:22.972997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534bea3-3abc-4717-ac5f-560a7f37dbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:22.973483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534bea3-3abc-4717-ac5f-560a7f37dbf5 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:22.974396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534bea3-3abc-4717-ac5f-560a7f37dbf5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:22.974950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534bea3-3abc-4717-ac5f-560a7f37dbf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:22.977080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534bea3-3abc-4717-ac5f-560a7f37dbf5 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:22.982789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 455/910] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:22.989447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf10cb-1b4a-44d0-9848-61dc7f0e550e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:23.137868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf10cb-1b4a-44d0-9848-61dc7f0e550e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:23.177654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523321d5-efb6-43e5-9a99-c2b4ce32db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:23.178335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523321d5-efb6-43e5-9a99-c2b4ce32db67 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:23.178613 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523321d5-efb6-43e5-9a99-c2b4ce32db67 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:23.178922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523321d5-efb6-43e5-9a99-c2b4ce32db67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:23.179732 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523321d5-efb6-43e5-9a99-c2b4ce32db67 None admin] RBAC: Adding query filter params (interface=public, service_id=2decab058a92458787f02d898b2af92a, region_id=RegionOne) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:23.180049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523321d5-efb6-43e5-9a99-c2b4ce32db67 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:23.181488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523321d5-efb6-43e5-9a99-c2b4ce32db67 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:23.187892 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 456/911] 10.222.0.79 () {56 vars in 1251 bytes} [Fri May 13 13:42:22 2022] GET /identity/v3/endpoints?service_id=2decab058a92458787f02d898b2af92a&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:23.193938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534bea3-3abc-4717-ac5f-560a7f37dbf5 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:23.312187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534bea3-3abc-4717-ac5f-560a7f37dbf5 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:23.349031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5188c-bec6-4661-9faa-485d463b256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:23.354758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5188c-bec6-4661-9faa-485d463b256c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:23.354758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5188c-bec6-4661-9faa-485d463b256c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:23.354758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5188c-bec6-4661-9faa-485d463b256c None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:23.354758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5188c-bec6-4661-9faa-485d463b256c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:23.354758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5188c-bec6-4661-9faa-485d463b256c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:23.359908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 456/912] 10.222.0.79 () {56 vars in 1087 bytes} [Fri May 13 13:42:23 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:42:24.961670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ae853-3a36-4b77-88d5-d9817236aced None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:24.962232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ae853-3a36-4b77-88d5-d9817236aced None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:24.962232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ae853-3a36-4b77-88d5-d9817236aced None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:24.963901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 457/913] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:24.969307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cfc6a-b72b-44db-bf7b-8caa8ad0a13a None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:24.969843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cfc6a-b72b-44db-bf7b-8caa8ad0a13a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:24.969843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cfc6a-b72b-44db-bf7b-8caa8ad0a13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:24.993352 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-7a7cfc6a-b72b-44db-bf7b-8caa8ad0a13a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:24.994839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7a7cfc6a-b72b-44db-bf7b-8caa8ad0a13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:25.067526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 457/914] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:25.092272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ae853-3a36-4b77-88d5-d9817236aced None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:25.215516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ae853-3a36-4b77-88d5-d9817236aced None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:25.253546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced3821-9e6e-4c8a-8818-9d99b3f7157f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:25.255113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced3821-9e6e-4c8a-8818-9d99b3f7157f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:25.255661 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced3821-9e6e-4c8a-8818-9d99b3f7157f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:25.256116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced3821-9e6e-4c8a-8818-9d99b3f7157f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:25.256922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced3821-9e6e-4c8a-8818-9d99b3f7157f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:25.258444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced3821-9e6e-4c8a-8818-9d99b3f7157f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-3ced3821-9e6e-4c8a-8818-9d99b3f7157f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:25.273148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:25.274411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:42:25.275422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:25.277873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 458/915] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:42:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:25.285490 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cfc6a-b72b-44db-bf7b-8caa8ad0a13a None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:25.420467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cfc6a-b72b-44db-bf7b-8caa8ad0a13a None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:25.466854 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bad089-41b2-4b67-8421-1fcaa093ff0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:25.467434 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad089-41b2-4b67-8421-1fcaa093ff0b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:25.467712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad089-41b2-4b67-8421-1fcaa093ff0b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:25.467982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bad089-41b2-4b67-8421-1fcaa093ff0b None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:25.468689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bad089-41b2-4b67-8421-1fcaa093ff0b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:25.468989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bad089-41b2-4b67-8421-1fcaa093ff0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:25.470373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bad089-41b2-4b67-8421-1fcaa093ff0b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:25.477413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 458/916] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:42:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:25.487516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced3821-9e6e-4c8a-8818-9d99b3f7157f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:25.628398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced3821-9e6e-4c8a-8818-9d99b3f7157f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:25.669570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b89e18-157a-4dcf-9690-2e308904fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:25.670654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b89e18-157a-4dcf-9690-2e308904fbc9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:25.671174 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b89e18-157a-4dcf-9690-2e308904fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:25.671652 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b89e18-157a-4dcf-9690-2e308904fbc9 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:25.672562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b89e18-157a-4dcf-9690-2e308904fbc9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:25.673068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b89e18-157a-4dcf-9690-2e308904fbc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:25.674727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b89e18-157a-4dcf-9690-2e308904fbc9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:25.680905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 459/917] 10.222.0.79 () {56 vars in 1114 bytes} [Fri May 13 13:42:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:25.685658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b767d-f79d-485c-a053-9afb0e2261cc None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:25.685658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b767d-f79d-485c-a053-9afb0e2261cc None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:25.685658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b767d-f79d-485c-a053-9afb0e2261cc None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:25.685658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 459/918] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:25.687162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b89e18-157a-4dcf-9690-2e308904fbc9 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:25.689215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911bc126-cde5-4e32-a552-f195f25c2f5c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:25.689495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911bc126-cde5-4e32-a552-f195f25c2f5c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:25.690300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911bc126-cde5-4e32-a552-f195f25c2f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:25.709141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-911bc126-cde5-4e32-a552-f195f25c2f5c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:25.710358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-911bc126-cde5-4e32-a552-f195f25c2f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:25.784247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 460/919] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:25.785590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9b130-52d5-4ce8-92cd-a54d20a5523d None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:25.785950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9b130-52d5-4ce8-92cd-a54d20a5523d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:25.786306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9b130-52d5-4ce8-92cd-a54d20a5523d None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:25.791148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 461/920] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:25.791148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0897516-ce84-4368-b35b-b2b905fa60ab None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:25.791148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0897516-ce84-4368-b35b-b2b905fa60ab None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:25.791148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0897516-ce84-4368-b35b-b2b905fa60ab None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:25.791148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 462/921] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:25.793267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bfb6c-dc7e-47c3-b939-753f4d21331f None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:25.793588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bfb6c-dc7e-47c3-b939-753f4d21331f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:25.793953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bfb6c-dc7e-47c3-b939-753f4d21331f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:25.811403 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-390bfb6c-dc7e-47c3-b939-753f4d21331f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:25.812708 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-390bfb6c-dc7e-47c3-b939-753f4d21331f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:25.820199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b89e18-157a-4dcf-9690-2e308904fbc9 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:25.856371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fed70-532a-4dfe-9abd-32c8cf8db797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:25.857019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fed70-532a-4dfe-9abd-32c8cf8db797 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:25.857309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fed70-532a-4dfe-9abd-32c8cf8db797 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:25.857559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fed70-532a-4dfe-9abd-32c8cf8db797 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:25.858307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fed70-532a-4dfe-9abd-32c8cf8db797 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:25.859703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fed70-532a-4dfe-9abd-32c8cf8db797 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:25.878968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 460/922] 10.222.0.79 () {60 vars in 1144 bytes} [Fri May 13 13:42:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:42:25.880638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4907cb-e6f4-42cb-a70f-0489f20fc369 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:25.880858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4907cb-e6f4-42cb-a70f-0489f20fc369 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:25.881146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4907cb-e6f4-42cb-a70f-0489f20fc369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:25.900302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-ec4907cb-e6f4-42cb-a70f-0489f20fc369 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:25.901558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ec4907cb-e6f4-42cb-a70f-0489f20fc369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:25.925827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 463/923] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:25.927610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bfb6c-dc7e-47c3-b939-753f4d21331f None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.000115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 461/924] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:26.001800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4907cb-e6f4-42cb-a70f-0489f20fc369 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.072636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bfb6c-dc7e-47c3-b939-753f4d21331f None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:26.116632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e00aca-fba5-4486-b4d0-bdf50517171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:26.117458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e00aca-fba5-4486-b4d0-bdf50517171f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:26.117734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e00aca-fba5-4486-b4d0-bdf50517171f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:26.118003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e00aca-fba5-4486-b4d0-bdf50517171f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:26.122892 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4907cb-e6f4-42cb-a70f-0489f20fc369 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:26.131719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e00aca-fba5-4486-b4d0-bdf50517171f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-f3e00aca-fba5-4486-b4d0-bdf50517171f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:26.139977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:26.141146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:26.142009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 464/925] 10.222.0.79 () {56 vars in 1120 bytes} [Fri May 13 13:42:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:26.143136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e00aca-fba5-4486-b4d0-bdf50517171f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.161202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc831e-6056-4970-800a-c4bed154bf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:26.162096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc831e-6056-4970-800a-c4bed154bf62 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:26.162374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc831e-6056-4970-800a-c4bed154bf62 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:26.162677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc831e-6056-4970-800a-c4bed154bf62 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:26.163849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc831e-6056-4970-800a-c4bed154bf62 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:26.165247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc831e-6056-4970-800a-c4bed154bf62 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-addc831e-6056-4970-800a-c4bed154bf62 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:26.178430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:26.179678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:26.180617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 462/926] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:26.181450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc831e-6056-4970-800a-c4bed154bf62 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.265828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e00aca-fba5-4486-b4d0-bdf50517171f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:26.301173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc831e-6056-4970-800a-c4bed154bf62 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:26.312933 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccd77e-b204-4e33-a4b8-86f1e5c29e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:26.313881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccd77e-b204-4e33-a4b8-86f1e5c29e78 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:26.314077 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccd77e-b204-4e33-a4b8-86f1e5c29e78 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:26.314783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccd77e-b204-4e33-a4b8-86f1e5c29e78 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:26.315303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccd77e-b204-4e33-a4b8-86f1e5c29e78 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:26.316686 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccd77e-b204-4e33-a4b8-86f1e5c29e78 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-afccd77e-b204-4e33-a4b8-86f1e5c29e78 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:26.329909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:26.331026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:26.331896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 465/927] 10.222.0.79 () {56 vars in 1132 bytes} [Fri May 13 13:42:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:26.333476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccd77e-b204-4e33-a4b8-86f1e5c29e78 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.343566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349fd8c-7b60-4455-859a-faaaa9a35721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:26.344285 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349fd8c-7b60-4455-859a-faaaa9a35721 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:26.344605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349fd8c-7b60-4455-859a-faaaa9a35721 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:26.344942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349fd8c-7b60-4455-859a-faaaa9a35721 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:26.346982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349fd8c-7b60-4455-859a-faaaa9a35721 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:26.347601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349fd8c-7b60-4455-859a-faaaa9a35721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:26.349351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349fd8c-7b60-4455-859a-faaaa9a35721 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:26.359799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 463/928] 10.222.0.79 () {56 vars in 1115 bytes} [Fri May 13 13:42:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:26.360657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349fd8c-7b60-4455-859a-faaaa9a35721 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.471314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccd77e-b204-4e33-a4b8-86f1e5c29e78 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:26.475975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349fd8c-7b60-4455-859a-faaaa9a35721 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:26.512005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165cdf9-cfdd-4222-9189-ad4198f48212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:26.512728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165cdf9-cfdd-4222-9189-ad4198f48212 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:26.513009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165cdf9-cfdd-4222-9189-ad4198f48212 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:26.513384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165cdf9-cfdd-4222-9189-ad4198f48212 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:26.514095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165cdf9-cfdd-4222-9189-ad4198f48212 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:26.514393 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165cdf9-cfdd-4222-9189-ad4198f48212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:26.514569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f2e9b-57e8-474e-a9aa-4f18a314e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:26.515192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f2e9b-57e8-474e-a9aa-4f18a314e002 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:26.515471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f2e9b-57e8-474e-a9aa-4f18a314e002 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:26.515744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f2e9b-57e8-474e-a9aa-4f18a314e002 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:26.515838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165cdf9-cfdd-4222-9189-ad4198f48212 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:26.516449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f2e9b-57e8-474e-a9aa-4f18a314e002 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:26.516926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f2e9b-57e8-474e-a9aa-4f18a314e002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:26.518756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f2e9b-57e8-474e-a9aa-4f18a314e002 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:26.521828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 464/929] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:26.522895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165cdf9-cfdd-4222-9189-ad4198f48212 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.529495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 466/930] 10.222.0.79 () {56 vars in 1122 bytes} [Fri May 13 13:42:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:26.530688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f2e9b-57e8-474e-a9aa-4f18a314e002 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.654032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f2e9b-57e8-474e-a9aa-4f18a314e002 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:26.662921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165cdf9-cfdd-4222-9189-ad4198f48212 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:26.695727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa5a7b-93c0-431c-8cfa-b33d5a266b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:26.696825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa5a7b-93c0-431c-8cfa-b33d5a266b86 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:26.697289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa5a7b-93c0-431c-8cfa-b33d5a266b86 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:26.697733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa5a7b-93c0-431c-8cfa-b33d5a266b86 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:26.699028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa5a7b-93c0-431c-8cfa-b33d5a266b86 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:26.699834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa5a7b-93c0-431c-8cfa-b33d5a266b86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:26.702375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa5a7b-93c0-431c-8cfa-b33d5a266b86 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:26.714757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 467/931] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:26.714757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa5a7b-93c0-431c-8cfa-b33d5a266b86 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.720836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1a715-48a5-46a7-94f6-1f85337861c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:26.721922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1a715-48a5-46a7-94f6-1f85337861c2 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:26.722372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1a715-48a5-46a7-94f6-1f85337861c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:26.722889 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1a715-48a5-46a7-94f6-1f85337861c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:26.739061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1a715-48a5-46a7-94f6-1f85337861c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:26.740699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1a715-48a5-46a7-94f6-1f85337861c2 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-f7b1a715-48a5-46a7-94f6-1f85337861c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:26.768338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:26.770332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:26.772467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:26.772467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:26.772467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:26.772467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:26.775273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 465/932] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:26.779485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1a715-48a5-46a7-94f6-1f85337861c2 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.871472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa5a7b-93c0-431c-8cfa-b33d5a266b86 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:26.920198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208c147-42be-4966-a95b-bf0edd3d8bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:26.920778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c147-42be-4966-a95b-bf0edd3d8bd2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:26.921059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c147-42be-4966-a95b-bf0edd3d8bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:26.921337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c147-42be-4966-a95b-bf0edd3d8bd2 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:26.922047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208c147-42be-4966-a95b-bf0edd3d8bd2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:26.922291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1a715-48a5-46a7-94f6-1f85337861c2 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:26.922444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208c147-42be-4966-a95b-bf0edd3d8bd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:26.923826 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208c147-42be-4966-a95b-bf0edd3d8bd2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:26.931873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 468/933] 10.222.0.79 () {56 vars in 1122 bytes} [Fri May 13 13:42:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:26.933270 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208c147-42be-4966-a95b-bf0edd3d8bd2 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:26.979896 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa084a8e-a014-474e-bf5c-8ccda451fb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:26.980670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084a8e-a014-474e-bf5c-8ccda451fb8a None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:26.980949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084a8e-a014-474e-bf5c-8ccda451fb8a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:26.981220 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084a8e-a014-474e-bf5c-8ccda451fb8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:26.981973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa084a8e-a014-474e-bf5c-8ccda451fb8a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:26.983390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa084a8e-a014-474e-bf5c-8ccda451fb8a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:27.006996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 466/934] 10.222.0.79 () {60 vars in 1144 bytes} [Fri May 13 13:42:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:42:27.007980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa084a8e-a014-474e-bf5c-8ccda451fb8a None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:27.125183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208c147-42be-4966-a95b-bf0edd3d8bd2 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:27.154193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa084a8e-a014-474e-bf5c-8ccda451fb8a None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:27.191112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d1a7f-82c9-4aa8-bc00-5844cb891e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:27.192567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d1a7f-82c9-4aa8-bc00-5844cb891e11 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:27.193052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d1a7f-82c9-4aa8-bc00-5844cb891e11 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:27.193526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d1a7f-82c9-4aa8-bc00-5844cb891e11 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:27.193795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3ae59-b47f-4e21-a9db-3bfd997767fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:27.194635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3ae59-b47f-4e21-a9db-3bfd997767fb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:27.194900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d1a7f-82c9-4aa8-bc00-5844cb891e11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:27.195246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3ae59-b47f-4e21-a9db-3bfd997767fb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:27.195430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d1a7f-82c9-4aa8-bc00-5844cb891e11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:27.195833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3ae59-b47f-4e21-a9db-3bfd997767fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:27.196784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3ae59-b47f-4e21-a9db-3bfd997767fb None admin] RBAC: Adding query filter params (interface=public, service_id=901a7a632f94456e9d1d98525ab8606a, region_id=RegionOne) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:27.197239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3ae59-b47f-4e21-a9db-3bfd997767fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:27.198033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d1a7f-82c9-4aa8-bc00-5844cb891e11 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:27.198954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3ae59-b47f-4e21-a9db-3bfd997767fb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:27.206216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 467/935] 10.222.0.79 () {56 vars in 1251 bytes} [Fri May 13 13:42:27 2022] GET /identity/v3/endpoints?service_id=901a7a632f94456e9d1d98525ab8606a&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:27.214795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3ae59-b47f-4e21-a9db-3bfd997767fb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:27.219236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 469/936] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:27.225321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d1a7f-82c9-4aa8-bc00-5844cb891e11 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:27.352168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d1a7f-82c9-4aa8-bc00-5844cb891e11 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:27.371167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3ae59-b47f-4e21-a9db-3bfd997767fb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:27.391149 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a15bfc-7011-4d75-91ea-37c0eea6ef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:27.392278 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a15bfc-7011-4d75-91ea-37c0eea6ef29 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:27.392733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a15bfc-7011-4d75-91ea-37c0eea6ef29 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:27.393759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a15bfc-7011-4d75-91ea-37c0eea6ef29 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:27.402693 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a15bfc-7011-4d75-91ea-37c0eea6ef29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:27.404488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a15bfc-7011-4d75-91ea-37c0eea6ef29 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:27.408722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecba40-77c6-4630-a03f-6ece06188eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:27.409488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecba40-77c6-4630-a03f-6ece06188eec None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:27.409841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecba40-77c6-4630-a03f-6ece06188eec None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:27.410193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecba40-77c6-4630-a03f-6ece06188eec None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:27.410993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecba40-77c6-4630-a03f-6ece06188eec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:27.412483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecba40-77c6-4630-a03f-6ece06188eec None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:27.419731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 468/937] 10.222.0.79 () {56 vars in 1087 bytes} [Fri May 13 13:42:27 2022] GET /identity/v3/services => generated 1497 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-55a15bfc-7011-4d75-91ea-37c0eea6ef29 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:27.424819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:27.425808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:27.427460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:27.427460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:27.427460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:27.427460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:27.427460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:27.427460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:27.427460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:27.427460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:27.427460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:42:27.427460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:27.430263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 470/938] 10.222.0.79 () {56 vars in 1117 bytes} [Fri May 13 13:42:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:27.436035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecba40-77c6-4630-a03f-6ece06188eec None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:27.559775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecba40-77c6-4630-a03f-6ece06188eec None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:27.598534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa047a-6289-412a-9101-9a5a97d909d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:27.599253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa047a-6289-412a-9101-9a5a97d909d8 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:27.599534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa047a-6289-412a-9101-9a5a97d909d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:27.599816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa047a-6289-412a-9101-9a5a97d909d8 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:27.600663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa047a-6289-412a-9101-9a5a97d909d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:27.600975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa047a-6289-412a-9101-9a5a97d909d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:27.602450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa047a-6289-412a-9101-9a5a97d909d8 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:27.621892 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 469/939] 10.222.0.79 () {56 vars in 1149 bytes} [Fri May 13 13:42:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:27.627534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a15bfc-7011-4d75-91ea-37c0eea6ef29 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:27.742647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a15bfc-7011-4d75-91ea-37c0eea6ef29 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:27.783093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cace58-fb9d-4893-bad9-9fe47150030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:27.784582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cace58-fb9d-4893-bad9-9fe47150030b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:27.784947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cace58-fb9d-4893-bad9-9fe47150030b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:27.785261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cace58-fb9d-4893-bad9-9fe47150030b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:27.800672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cace58-fb9d-4893-bad9-9fe47150030b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:27.802197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cace58-fb9d-4893-bad9-9fe47150030b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-02cace58-fb9d-4893-bad9-9fe47150030b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:27.826082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:27.827069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:27.828009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:27.828009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:27.828009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:27.828009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:27.828009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 471/940] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:27.835274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa047a-6289-412a-9101-9a5a97d909d8 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:27.968171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa047a-6289-412a-9101-9a5a97d909d8 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:28.006786 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc6dbd-dfe8-4a2a-b9c6-5aca2b7b515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:28.007495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc6dbd-dfe8-4a2a-b9c6-5aca2b7b515b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:28.008313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc6dbd-dfe8-4a2a-b9c6-5aca2b7b515b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:28.008313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc6dbd-dfe8-4a2a-b9c6-5aca2b7b515b None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:28.008815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc6dbd-dfe8-4a2a-b9c6-5aca2b7b515b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:28.009122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc6dbd-dfe8-4a2a-b9c6-5aca2b7b515b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:28.010541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc6dbd-dfe8-4a2a-b9c6-5aca2b7b515b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:28.026225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 470/941] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:28.035841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cace58-fb9d-4893-bad9-9fe47150030b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:28.156308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cace58-fb9d-4893-bad9-9fe47150030b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:28.203952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4d81e-263e-4dd5-be11-2aa9248aa145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:28.205824 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4d81e-263e-4dd5-be11-2aa9248aa145 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:28.206301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4d81e-263e-4dd5-be11-2aa9248aa145 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:28.206832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4d81e-263e-4dd5-be11-2aa9248aa145 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:28.219434 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4d81e-263e-4dd5-be11-2aa9248aa145 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:28.222673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4d81e-263e-4dd5-be11-2aa9248aa145 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-59a4d81e-263e-4dd5-be11-2aa9248aa145 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:28.245203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:28.247438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:28.249609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:28.249609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:28.249609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:28.249609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:28.249609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:28.249609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:28.249609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:28.249609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:28.249609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 472/942] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:28.256904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc6dbd-dfe8-4a2a-b9c6-5aca2b7b515b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:28.371314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc6dbd-dfe8-4a2a-b9c6-5aca2b7b515b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:28.423585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d59083-6313-4d8d-ad52-c14dd2a39b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:28.424532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59083-6313-4d8d-ad52-c14dd2a39b0a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:28.424957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59083-6313-4d8d-ad52-c14dd2a39b0a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:28.425375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d59083-6313-4d8d-ad52-c14dd2a39b0a None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:28.426642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d59083-6313-4d8d-ad52-c14dd2a39b0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:28.427213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d59083-6313-4d8d-ad52-c14dd2a39b0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:28.429560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d59083-6313-4d8d-ad52-c14dd2a39b0a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:28.447471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 471/943] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:28.456722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4d81e-263e-4dd5-be11-2aa9248aa145 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:28.570415 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4d81e-263e-4dd5-be11-2aa9248aa145 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:28.611461 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c2c3b-1660-4a19-8679-a85e2ffb057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:28.612508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c2c3b-1660-4a19-8679-a85e2ffb057f None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:28.612867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c2c3b-1660-4a19-8679-a85e2ffb057f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:28.613145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c2c3b-1660-4a19-8679-a85e2ffb057f None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/598c323e8ed5455cbe4a568b360d8eab/roles/f4a90110e2c845408f5ec7685f7ef2d0` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:28.615534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c2c3b-1660-4a19-8679-a85e2ffb057f None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=598c323e8ed5455cbe4a568b360d8eab, role_id=f4a90110e2c845408f5ec7685f7ef2d0)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:28.617146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c2c3b-1660-4a19-8679-a85e2ffb057f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:28.633429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 473/944] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:42:28 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/598c323e8ed5455cbe4a568b360d8eab/roles/f4a90110e2c845408f5ec7685f7ef2d0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:42:28.729965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0e66c-788f-445e-a300-0015c9cceacd None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:28.730343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0e66c-788f-445e-a300-0015c9cceacd None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:28.730511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0e66c-788f-445e-a300-0015c9cceacd None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:28.732286 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 472/945] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:28.738879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62803c7b-da06-4e8a-87b9-9f0a5bb06a5a None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:28.739252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62803c7b-da06-4e8a-87b9-9f0a5bb06a5a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:28.739523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62803c7b-da06-4e8a-87b9-9f0a5bb06a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:28.758492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-62803c7b-da06-4e8a-87b9-9f0a5bb06a5a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:28.759747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-62803c7b-da06-4e8a-87b9-9f0a5bb06a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:28.854990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 474/946] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:28.883441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0e66c-788f-445e-a300-0015c9cceacd None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:28.999530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0e66c-788f-445e-a300-0015c9cceacd None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:29.055182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:29.056193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:29.056653 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:29.057096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:29.058254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:29.060692 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:29.062978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WARNING keystone.server.flask.common [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 13 13:42:29.098572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.sql.core [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 13 13:42:29.098572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 13 13:42:29.098572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [parameters: {'id': '0743652736df4356b593a3bc0272cebf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 13 13:42:29.098572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0743652736df4356b593a3bc0272cebf.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0743652736df4356b593a3bc0272cebf. May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application conn.query(q) May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:42:29.128122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application result.read() May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 13 13:42:29.129022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return project_ref.to_dict() May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application next(self.gen) May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application yield resource May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application next(self.gen) May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self.session.rollback() May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self.force_reraise() May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise self.value May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application session.commit() May 13 13:42:29.130016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._prepare_impl() May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self.session.flush() May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._flush(objects) May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application compat.raise_( May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application flush_context.execute() May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 13 13:42:29.131736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rec.execute(self) May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application _emit_insert_statements( May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application c = connection._execute_20( May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ret = self._execute_context( May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 13 13:42:29.133830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application conn.query(q) May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application result.read() May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:42:29.135656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [parameters: {'id': '0743652736df4356b593a3bc0272cebf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:29.137651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:29.143322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:29.145502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:29.145502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 13 13:42:29.145502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 13 13:42:29.145502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:29.145502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:29.145502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 13 13:42:29.145502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.Conflict( May 13 13:42:29.145502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0743652736df4356b593a3bc0272cebf. May 13 13:42:29.145502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:29.145502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 473/947] 10.222.0.79 () {60 vars in 1140 bytes} [Fri May 13 13:42:28 2022] POST /identity/v3/projects => generated 262 bytes in 249 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:29.146893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62803c7b-da06-4e8a-87b9-9f0a5bb06a5a None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:29.259725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62803c7b-da06-4e8a-87b9-9f0a5bb06a5a None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:29.301228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c8b60-6ac4-456d-b08e-dab7ee464423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:29.302444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8b60-6ac4-456d-b08e-dab7ee464423 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:29.302979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8b60-6ac4-456d-b08e-dab7ee464423 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:29.303485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8b60-6ac4-456d-b08e-dab7ee464423 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:29.312209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c8b60-6ac4-456d-b08e-dab7ee464423 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:29.314060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c8b60-6ac4-456d-b08e-dab7ee464423 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-d93c8b60-6ac4-456d-b08e-dab7ee464423 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:29.331351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:29.332303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:29.333203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:29.333203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:29.333203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:29.333203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:29.333203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:29.333203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:29.333203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:29.333203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:29.335799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 475/948] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:29.346768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:29.470483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490acde-0711-4526-8a33-761ea582ca56 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:29.526203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eb1ef-b06a-4be7-8d05-00db6382aac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:29.527619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb1ef-b06a-4be7-8d05-00db6382aac4 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:29.528291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb1ef-b06a-4be7-8d05-00db6382aac4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:29.528769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb1ef-b06a-4be7-8d05-00db6382aac4 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:29.529684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb1ef-b06a-4be7-8d05-00db6382aac4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:29.530162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb1ef-b06a-4be7-8d05-00db6382aac4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:29.532298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb1ef-b06a-4be7-8d05-00db6382aac4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:29.557360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 474/949] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:29.786125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfafcdc-b033-416d-aa55-4158d1080f79 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:29.786462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfafcdc-b033-416d-aa55-4158d1080f79 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:29.786583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfafcdc-b033-416d-aa55-4158d1080f79 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:29.788140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 476/950] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:29.793507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fae07f-9647-4d23-ade7-e8fdd87d6f6a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:29.793815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fae07f-9647-4d23-ade7-e8fdd87d6f6a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:29.794072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fae07f-9647-4d23-ade7-e8fdd87d6f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:29.813769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-53fae07f-9647-4d23-ade7-e8fdd87d6f6a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:29.815426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-53fae07f-9647-4d23-ade7-e8fdd87d6f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:29.888460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 475/951] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:29.918792 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfafcdc-b033-416d-aa55-4158d1080f79 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:30.042592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfafcdc-b033-416d-aa55-4158d1080f79 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:30.081504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb070a2-ac03-4306-9eaf-703607fbd962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:30.082538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb070a2-ac03-4306-9eaf-703607fbd962 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:30.082941 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb070a2-ac03-4306-9eaf-703607fbd962 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:30.083292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb070a2-ac03-4306-9eaf-703607fbd962 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:30.084295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb070a2-ac03-4306-9eaf-703607fbd962 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:30.086519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb070a2-ac03-4306-9eaf-703607fbd962 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-bcb070a2-ac03-4306-9eaf-703607fbd962 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:30.100491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:30.101616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:30.102447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 477/952] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:30.112095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fae07f-9647-4d23-ade7-e8fdd87d6f6a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:30.200514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1adbb-1f9e-4d62-892b-53b88726c9c9 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:30.200774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1adbb-1f9e-4d62-892b-53b88726c9c9 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:30.201114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1adbb-1f9e-4d62-892b-53b88726c9c9 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:30.202599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 478/953] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:30.208583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc96c-cb00-4d31-a5bd-4d9b58f47f20 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:30.208905 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc96c-cb00-4d31-a5bd-4d9b58f47f20 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:30.209140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc96c-cb00-4d31-a5bd-4d9b58f47f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:30.228176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-5bcbc96c-cb00-4d31-a5bd-4d9b58f47f20 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:30.229374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5bcbc96c-cb00-4d31-a5bd-4d9b58f47f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:30.245202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fae07f-9647-4d23-ade7-e8fdd87d6f6a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:30.311681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad5873-acb2-4c4b-86dd-2fe499941428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:30.312892 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5873-acb2-4c4b-86dd-2fe499941428 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:30.313527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5873-acb2-4c4b-86dd-2fe499941428 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:30.314075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5873-acb2-4c4b-86dd-2fe499941428 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:30.315438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 479/954] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:30.316261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad5873-acb2-4c4b-86dd-2fe499941428 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:30.316841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad5873-acb2-4c4b-86dd-2fe499941428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:30.318856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad5873-acb2-4c4b-86dd-2fe499941428 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:30.325965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 476/955] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:30.334699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbc96c-cb00-4d31-a5bd-4d9b58f47f20 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:30.335216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad5873-acb2-4c4b-86dd-2fe499941428 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:30.472555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad5873-acb2-4c4b-86dd-2fe499941428 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:30.476695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbc96c-cb00-4d31-a5bd-4d9b58f47f20 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:30.517029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d750705-78da-4cd9-81ea-126d50039110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:30.519104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d750705-78da-4cd9-81ea-126d50039110 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:30.519418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d750705-78da-4cd9-81ea-126d50039110 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:30.519694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d750705-78da-4cd9-81ea-126d50039110 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:30.520453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d750705-78da-4cd9-81ea-126d50039110 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:30.521994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d750705-78da-4cd9-81ea-126d50039110 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:30.524980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c55366-29d1-4221-928e-9a8bb72e485a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:30.526233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c55366-29d1-4221-928e-9a8bb72e485a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:30.526743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c55366-29d1-4221-928e-9a8bb72e485a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:30.527266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c55366-29d1-4221-928e-9a8bb72e485a None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:30.528205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c55366-29d1-4221-928e-9a8bb72e485a None admin] RBAC: Adding query filter params (type=compute) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:30.528807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c55366-29d1-4221-928e-9a8bb72e485a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:30.530429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c55366-29d1-4221-928e-9a8bb72e485a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-6d750705-78da-4cd9-81ea-126d50039110 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:30.536520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:30.537601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:30.538610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:30.546976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 477/957] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:30.546976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d750705-78da-4cd9-81ea-126d50039110 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:30.547613 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 480/957] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:30.675331 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d750705-78da-4cd9-81ea-126d50039110 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:30.715866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5145b60-f336-44fa-b677-1fe74c5fef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:30.716553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5145b60-f336-44fa-b677-1fe74c5fef39 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:30.716836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5145b60-f336-44fa-b677-1fe74c5fef39 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:30.717113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5145b60-f336-44fa-b677-1fe74c5fef39 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:30.717833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5145b60-f336-44fa-b677-1fe74c5fef39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:30.718140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5145b60-f336-44fa-b677-1fe74c5fef39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:30.719595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5145b60-f336-44fa-b677-1fe74c5fef39 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:30.726414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 478/958] 10.222.0.79 () {56 vars in 1122 bytes} [Fri May 13 13:42:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:30.732398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c55366-29d1-4221-928e-9a8bb72e485a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:30.856938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c55366-29d1-4221-928e-9a8bb72e485a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:30.926032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f5c1-64b3-40c6-860d-1577358dec0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:30.926611 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f5c1-64b3-40c6-860d-1577358dec0a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:30.926930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f5c1-64b3-40c6-860d-1577358dec0a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:30.927176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f5c1-64b3-40c6-860d-1577358dec0a None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:30.927915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f5c1-64b3-40c6-860d-1577358dec0a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:30.928212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f5c1-64b3-40c6-860d-1577358dec0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:30.930050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f5c1-64b3-40c6-860d-1577358dec0a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:30.936579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 481/959] 10.222.0.79 () {56 vars in 1122 bytes} [Fri May 13 13:42:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:30.945799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5145b60-f336-44fa-b677-1fe74c5fef39 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:31.103018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5145b60-f336-44fa-b677-1fe74c5fef39 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:31.147522 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a021509-94bc-47a9-8048-657b37aca4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:31.148181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a021509-94bc-47a9-8048-657b37aca4e1 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:31.148461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a021509-94bc-47a9-8048-657b37aca4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:31.148741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a021509-94bc-47a9-8048-657b37aca4e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:31.149503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a021509-94bc-47a9-8048-657b37aca4e1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:31.150913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a021509-94bc-47a9-8048-657b37aca4e1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:31.179410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 479/960] 10.222.0.79 () {60 vars in 1144 bytes} [Fri May 13 13:42:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:42:31.275926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb4e2f-debe-4ca3-9311-c911391e0280 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:31.276747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb4e2f-debe-4ca3-9311-c911391e0280 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:31.277093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb4e2f-debe-4ca3-9311-c911391e0280 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:31.279112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 482/961] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:31.297158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc840f90-563f-46b7-a2c7-86ae019cf31a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:31.297650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc840f90-563f-46b7-a2c7-86ae019cf31a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:31.298123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc840f90-563f-46b7-a2c7-86ae019cf31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:31.320267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-bc840f90-563f-46b7-a2c7-86ae019cf31a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:31.322290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-bc840f90-563f-46b7-a2c7-86ae019cf31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:31.427057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 480/962] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:31.452090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb4e2f-debe-4ca3-9311-c911391e0280 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:31.566353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb4e2f-debe-4ca3-9311-c911391e0280 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:31.606531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6bb02-99b7-468a-b344-417af4263367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:31.607399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6bb02-99b7-468a-b344-417af4263367 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:31.607684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6bb02-99b7-468a-b344-417af4263367 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:31.607960 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6bb02-99b7-468a-b344-417af4263367 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:31.619669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6bb02-99b7-468a-b344-417af4263367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-9ec6bb02-99b7-468a-b344-417af4263367 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:31.628712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:31.629721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:31.630582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 483/963] 10.222.0.79 () {56 vars in 1120 bytes} [Fri May 13 13:42:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:31.636212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc840f90-563f-46b7-a2c7-86ae019cf31a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:31.755700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc840f90-563f-46b7-a2c7-86ae019cf31a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:31.792120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91fd37-caa4-46cc-8080-d4d331dde8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:31.792795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91fd37-caa4-46cc-8080-d4d331dde8f0 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:31.793078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91fd37-caa4-46cc-8080-d4d331dde8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:31.793317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91fd37-caa4-46cc-8080-d4d331dde8f0 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:31.794107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91fd37-caa4-46cc-8080-d4d331dde8f0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:31.794407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91fd37-caa4-46cc-8080-d4d331dde8f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:31.795815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91fd37-caa4-46cc-8080-d4d331dde8f0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:31.805552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 481/964] 10.222.0.79 () {56 vars in 1115 bytes} [Fri May 13 13:42:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:31.811164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6bb02-99b7-468a-b344-417af4263367 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:31.920074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6bb02-99b7-468a-b344-417af4263367 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:31.961391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84bc35-55ce-43cd-88cd-f24451a9022d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:31.961391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84bc35-55ce-43cd-88cd-f24451a9022d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:31.962058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84bc35-55ce-43cd-88cd-f24451a9022d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:31.962058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84bc35-55ce-43cd-88cd-f24451a9022d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:31.973991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84bc35-55ce-43cd-88cd-f24451a9022d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:31.975309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84bc35-55ce-43cd-88cd-f24451a9022d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-5c84bc35-55ce-43cd-88cd-f24451a9022d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:31.995529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:31.997684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:31.999218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:31.999218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:31.999218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:31.999218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:31.999218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 484/965] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:32.006239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91fd37-caa4-46cc-8080-d4d331dde8f0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:32.146267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91fd37-caa4-46cc-8080-d4d331dde8f0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:32.186289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd7b9a-8d17-42c9-84d6-fbca07fcf008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:32.187024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd7b9a-8d17-42c9-84d6-fbca07fcf008 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:32.187418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd7b9a-8d17-42c9-84d6-fbca07fcf008 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:32.187698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd7b9a-8d17-42c9-84d6-fbca07fcf008 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:32.188470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd7b9a-8d17-42c9-84d6-fbca07fcf008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:32.188778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd7b9a-8d17-42c9-84d6-fbca07fcf008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:32.191591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd7b9a-8d17-42c9-84d6-fbca07fcf008 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:32.206937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 482/966] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:32.212930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84bc35-55ce-43cd-88cd-f24451a9022d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:32.351891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84bc35-55ce-43cd-88cd-f24451a9022d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:32.388568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a5361-aa25-4896-8410-0424e44563a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:32.389393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a5361-aa25-4896-8410-0424e44563a8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:32.389679 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a5361-aa25-4896-8410-0424e44563a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:32.389954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a5361-aa25-4896-8410-0424e44563a8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:32.399171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a5361-aa25-4896-8410-0424e44563a8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:32.400632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a5361-aa25-4896-8410-0424e44563a8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-c79a5361-aa25-4896-8410-0424e44563a8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:32.416419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:32.418648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:32.420591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 485/967] 10.222.0.79 () {56 vars in 1117 bytes} [Fri May 13 13:42:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:32.430293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748fe2e-c05a-4de3-9d5e-cb3388b1fe1b None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:32.431212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748fe2e-c05a-4de3-9d5e-cb3388b1fe1b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:32.431959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748fe2e-c05a-4de3-9d5e-cb3388b1fe1b None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:32.433740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a5361-aa25-4896-8410-0424e44563a8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:32.435878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 483/968] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:32.446161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784b52b-1ae5-4919-8d12-b668e62636d5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:32.446824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784b52b-1ae5-4919-8d12-b668e62636d5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:32.447320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784b52b-1ae5-4919-8d12-b668e62636d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:32.468490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-d784b52b-1ae5-4919-8d12-b668e62636d5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:32.470308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d784b52b-1ae5-4919-8d12-b668e62636d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:32.564640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 484/969] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:32.592415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784b52b-1ae5-4919-8d12-b668e62636d5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:32.601545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a5361-aa25-4896-8410-0424e44563a8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:32.660251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc558-1586-44ec-9ace-07411fc529da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:32.661263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc558-1586-44ec-9ace-07411fc529da None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:32.661738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc558-1586-44ec-9ace-07411fc529da None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:32.662205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc558-1586-44ec-9ace-07411fc529da None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:32.663772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dc558-1586-44ec-9ace-07411fc529da None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:32.664385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dc558-1586-44ec-9ace-07411fc529da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:32.667131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dc558-1586-44ec-9ace-07411fc529da None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:32.701283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 486/970] 10.222.0.79 () {56 vars in 1149 bytes} [Fri May 13 13:42:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:32.709236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc558-1586-44ec-9ace-07411fc529da None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:32.738328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784b52b-1ae5-4919-8d12-b668e62636d5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:32.820446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77dc7da-d6d4-4007-a595-7f2b8ab4c69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:32.822350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dc7da-d6d4-4007-a595-7f2b8ab4c69d None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:32.823148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dc7da-d6d4-4007-a595-7f2b8ab4c69d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:32.823814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dc7da-d6d4-4007-a595-7f2b8ab4c69d None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:32.825646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77dc7da-d6d4-4007-a595-7f2b8ab4c69d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:32.830766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77dc7da-d6d4-4007-a595-7f2b8ab4c69d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:32.835985 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc558-1586-44ec-9ace-07411fc529da None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:32.853830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.sql.core [None req-d77dc7da-d6d4-4007-a595-7f2b8ab4c69d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 13 13:42:32.853830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 13 13:42:32.853830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [parameters: {'user_id': 'e5450adac11a412f8752c4f96d51b61f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 13 13:42:32.853830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-d77dc7da-d6d4-4007-a595-7f2b8ab4c69d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application conn.query(q) May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application result.read() May 13 13:42:32.864530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:42:32.866788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application next(self.gen) May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application yield resource May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application next(self.gen) May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self.session.rollback() May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self.force_reraise() May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise self.value May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application session.commit() May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 13 13:42:32.868275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._prepare_impl() May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self.session.flush() May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._flush(objects) May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application compat.raise_( May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application flush_context.execute() May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rec.execute(self) May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 13 13:42:32.869540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application _emit_insert_statements( May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application result = connection._execute_20( May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ret = self._execute_context( May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:42:32.870788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application conn.query(q) May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application result.read() May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:42:32.871831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e5450adac11a412f8752c4f96d51b61f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:32.872957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 13 13:42:32.873979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 13 13:42:32.881069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 13 13:42:32.881069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 13 13:42:32.881069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 13 13:42:32.881069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 13 13:42:32.881069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 13 13:42:32.881069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:32.881069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 485/971] 10.222.0.79 () {60 vars in 1131 bytes} [Fri May 13 13:42:32 2022] POST /identity/v3/users => generated 160 bytes in 275 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:32.884306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77dc7da-d6d4-4007-a595-7f2b8ab4c69d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:32.892154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c55582-2fe6-4b8a-946c-a7be1fc186e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:32.892859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55582-2fe6-4b8a-946c-a7be1fc186e5 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:32.893175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55582-2fe6-4b8a-946c-a7be1fc186e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:32.893458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55582-2fe6-4b8a-946c-a7be1fc186e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.027923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c55582-2fe6-4b8a-946c-a7be1fc186e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:33.031695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c55582-2fe6-4b8a-946c-a7be1fc186e5 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:33.031913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77dc7da-d6d4-4007-a595-7f2b8ab4c69d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:33.081588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176037ae-47ef-4319-8ae8-23ea777cc8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:33.082419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176037ae-47ef-4319-8ae8-23ea777cc8ee None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:33.082725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176037ae-47ef-4319-8ae8-23ea777cc8ee None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:33.083031 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176037ae-47ef-4319-8ae8-23ea777cc8ee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.093697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176037ae-47ef-4319-8ae8-23ea777cc8ee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:33.096730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176037ae-47ef-4319-8ae8-23ea777cc8ee None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-176037ae-47ef-4319-8ae8-23ea777cc8ee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:33.122321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:33.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:33.124577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:33.124577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:33.124577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:33.124577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:33.124577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:33.124577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:33.124577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:33.124577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:33.124577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:42:33.124577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:33.129421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 486/972] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:42:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:33.131954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176037ae-47ef-4319-8ae8-23ea777cc8ee None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:33.194991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 487/973] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:42:32 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:42:33.202228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c55582-2fe6-4b8a-946c-a7be1fc186e5 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:33.282586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176037ae-47ef-4319-8ae8-23ea777cc8ee None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:33.325745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267c037-5da0-45ae-ac42-619537baa471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:33.327444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267c037-5da0-45ae-ac42-619537baa471 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:33.328207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267c037-5da0-45ae-ac42-619537baa471 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:33.328714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267c037-5da0-45ae-ac42-619537baa471 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.329755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267c037-5da0-45ae-ac42-619537baa471 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:33.330346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267c037-5da0-45ae-ac42-619537baa471 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:33.332112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267c037-5da0-45ae-ac42-619537baa471 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:33.350345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c55582-2fe6-4b8a-946c-a7be1fc186e5 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:33.386884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 487/974] 10.222.0.79 () {56 vars in 1101 bytes} [Fri May 13 13:42:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:33.390192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4bbd1-f9ff-43a4-8b83-d77a4af4d9b4 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:33.390646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4bbd1-f9ff-43a4-8b83-d77a4af4d9b4 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:33.390993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4bbd1-f9ff-43a4-8b83-d77a4af4d9b4 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.394067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfc5f6-babf-4254-b2fb-a5b931583d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:33.395184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfc5f6-babf-4254-b2fb-a5b931583d08 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:33.395636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfc5f6-babf-4254-b2fb-a5b931583d08 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:33.396075 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfc5f6-babf-4254-b2fb-a5b931583d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.397605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 488/975] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:33.399427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d13802-d45b-48d2-ab08-47606a0f1bad None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:33.399715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d13802-d45b-48d2-ab08-47606a0f1bad None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:33.400020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d13802-d45b-48d2-ab08-47606a0f1bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.417217 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfc5f6-babf-4254-b2fb-a5b931583d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:33.420195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfc5f6-babf-4254-b2fb-a5b931583d08 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:33.420667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-03d13802-d45b-48d2-ab08-47606a0f1bad None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:33.422288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-03d13802-d45b-48d2-ab08-47606a0f1bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-44cfc5f6-babf-4254-b2fb-a5b931583d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:33.450106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:33.451398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:33.452468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:33.452468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:33.452468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:33.452468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:33.461005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 488/976] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:33.464108 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfc5f6-babf-4254-b2fb-a5b931583d08 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:33.531071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 489/977] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:33.557860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d13802-d45b-48d2-ab08-47606a0f1bad None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:33.610572 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfc5f6-babf-4254-b2fb-a5b931583d08 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:33.666122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bc368-8d1d-468c-a146-c690d0c31205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:33.666845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bc368-8d1d-468c-a146-c690d0c31205 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:33.667130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bc368-8d1d-468c-a146-c690d0c31205 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:33.667409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bc368-8d1d-468c-a146-c690d0c31205 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.668148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bc368-8d1d-468c-a146-c690d0c31205 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:33.668453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bc368-8d1d-468c-a146-c690d0c31205 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:33.669837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bc368-8d1d-468c-a146-c690d0c31205 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:33.685527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 489/978] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:33.694757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bc368-8d1d-468c-a146-c690d0c31205 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:33.717537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d13802-d45b-48d2-ab08-47606a0f1bad None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:33.763279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74509e-abb3-4bb5-b9db-9f6150ade913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:33.763989 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74509e-abb3-4bb5-b9db-9f6150ade913 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:33.764267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74509e-abb3-4bb5-b9db-9f6150ade913 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:33.764538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74509e-abb3-4bb5-b9db-9f6150ade913 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.765286 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74509e-abb3-4bb5-b9db-9f6150ade913 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:33.766765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74509e-abb3-4bb5-b9db-9f6150ade913 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:33.792304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 490/979] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:42:33 2022] POST /identity/v3/services => generated 242 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:42:33.833186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bc368-8d1d-468c-a146-c690d0c31205 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:33.882165 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0fb8c-cae0-4236-a95d-92bcf210aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:33.883018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fb8c-cae0-4236-a95d-92bcf210aad2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:33.883304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fb8c-cae0-4236-a95d-92bcf210aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:33.883580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fb8c-cae0-4236-a95d-92bcf210aad2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.891354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0fb8c-cae0-4236-a95d-92bcf210aad2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:33.892892 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0fb8c-cae0-4236-a95d-92bcf210aad2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-b9f0fb8c-cae0-4236-a95d-92bcf210aad2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:33.914755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:33.915915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:33.917061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:33.917061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:33.917061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:33.917061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:33.917061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:33.917061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:33.917061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:33.917061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:33.917061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 490/980] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:33.920544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74509e-abb3-4bb5-b9db-9f6150ade913 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:33.961679 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792739-6444-435e-b7a5-8f8cb0e32a66 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:33.961959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792739-6444-435e-b7a5-8f8cb0e32a66 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:33.962232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792739-6444-435e-b7a5-8f8cb0e32a66 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.964565 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 491/981] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:33.969770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbb7bc-123f-4795-bfda-03c0adfddf2e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:33.970064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbb7bc-123f-4795-bfda-03c0adfddf2e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:33.970326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbb7bc-123f-4795-bfda-03c0adfddf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:33.997752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-68bbb7bc-123f-4795-bfda-03c0adfddf2e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:33.999012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-68bbb7bc-123f-4795-bfda-03c0adfddf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:34.047335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74509e-abb3-4bb5-b9db-9f6150ade913 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:34.081857 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 492/982] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:34.085810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9077a-10ad-4cf8-b46c-a4171f6bcfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:34.086517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9077a-10ad-4cf8-b46c-a4171f6bcfbb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:34.086933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9077a-10ad-4cf8-b46c-a4171f6bcfbb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:34.087225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9077a-10ad-4cf8-b46c-a4171f6bcfbb None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:34.088834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9077a-10ad-4cf8-b46c-a4171f6bcfbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:34.088834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9077a-10ad-4cf8-b46c-a4171f6bcfbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:34.089910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9077a-10ad-4cf8-b46c-a4171f6bcfbb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:34.106765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 491/983] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:34.108339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbb7bc-123f-4795-bfda-03c0adfddf2e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:34.109480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9077a-10ad-4cf8-b46c-a4171f6bcfbb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:34.232782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9077a-10ad-4cf8-b46c-a4171f6bcfbb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:34.250575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbb7bc-123f-4795-bfda-03c0adfddf2e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:34.284620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45b5cd-9b1f-4545-9fec-4c87bd7f6b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:34.285591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45b5cd-9b1f-4545-9fec-4c87bd7f6b54 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:34.285591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45b5cd-9b1f-4545-9fec-4c87bd7f6b54 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:34.285850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45b5cd-9b1f-4545-9fec-4c87bd7f6b54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:34.286684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45b5cd-9b1f-4545-9fec-4c87bd7f6b54 None admin] RBAC: Adding query filter params (role.id=f4a90110e2c845408f5ec7685f7ef2d0, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=598c323e8ed5455cbe4a568b360d8eab) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:34.287024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45b5cd-9b1f-4545-9fec-4c87bd7f6b54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:34.289473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45b5cd-9b1f-4545-9fec-4c87bd7f6b54 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:34.299875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 492/984] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:42:34 2022] GET /identity/v3/role_assignments?user.id=598c323e8ed5455cbe4a568b360d8eab&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=f4a90110e2c845408f5ec7685f7ef2d0 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:34.300123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c18282-c153-4477-b758-b1ac8fdb4024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:34.300824 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c18282-c153-4477-b758-b1ac8fdb4024 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:34.301306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c18282-c153-4477-b758-b1ac8fdb4024 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:34.301794 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c18282-c153-4477-b758-b1ac8fdb4024 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:34.302825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c18282-c153-4477-b758-b1ac8fdb4024 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:34.304602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c18282-c153-4477-b758-b1ac8fdb4024 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-67c18282-c153-4477-b758-b1ac8fdb4024 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:34.327424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:34.329061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:34.330599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:34.339312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 493/985] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:34.341569 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45b5cd-9b1f-4545-9fec-4c87bd7f6b54 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:34.502853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45b5cd-9b1f-4545-9fec-4c87bd7f6b54 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:34.549457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a21a98-3289-4029-8d8c-a6efcc2c5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:34.550478 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a21a98-3289-4029-8d8c-a6efcc2c5aee None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:34.550981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a21a98-3289-4029-8d8c-a6efcc2c5aee None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:34.551446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a21a98-3289-4029-8d8c-a6efcc2c5aee None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:34.552390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a21a98-3289-4029-8d8c-a6efcc2c5aee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:34.552890 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a21a98-3289-4029-8d8c-a6efcc2c5aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:34.554506 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a21a98-3289-4029-8d8c-a6efcc2c5aee None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:34.565584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 493/986] 10.222.0.79 () {56 vars in 1122 bytes} [Fri May 13 13:42:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:34.574236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c18282-c153-4477-b758-b1ac8fdb4024 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:34.709881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c18282-c153-4477-b758-b1ac8fdb4024 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:34.775071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e61f51-8131-4263-9d57-cbccee3e1c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:34.776150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e61f51-8131-4263-9d57-cbccee3e1c14 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:34.776605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e61f51-8131-4263-9d57-cbccee3e1c14 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:34.777079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e61f51-8131-4263-9d57-cbccee3e1c14 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:34.778402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e61f51-8131-4263-9d57-cbccee3e1c14 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:34.778953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e61f51-8131-4263-9d57-cbccee3e1c14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:34.781486 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e61f51-8131-4263-9d57-cbccee3e1c14 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:34.794824 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 494/987] 10.222.0.79 () {56 vars in 1122 bytes} [Fri May 13 13:42:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:34.800004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a21a98-3289-4029-8d8c-a6efcc2c5aee None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:34.946609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a21a98-3289-4029-8d8c-a6efcc2c5aee None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:34.986333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37929600-d340-461a-9aa1-18bbc127a0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:34.987320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929600-d340-461a-9aa1-18bbc127a0c7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:34.987792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929600-d340-461a-9aa1-18bbc127a0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:34.988275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929600-d340-461a-9aa1-18bbc127a0c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:34.989261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37929600-d340-461a-9aa1-18bbc127a0c7 None admin] RBAC: Adding query filter params (interface=admin, service_id=901a7a632f94456e9d1d98525ab8606a, region_id=RegionOne) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:34.989758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37929600-d340-461a-9aa1-18bbc127a0c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:34.991352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37929600-d340-461a-9aa1-18bbc127a0c7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:35.001929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 494/988] 10.222.0.79 () {56 vars in 1249 bytes} [Fri May 13 13:42:34 2022] GET /identity/v3/endpoints?service_id=901a7a632f94456e9d1d98525ab8606a&interface=admin®ion_id=RegionOne => generated 193 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:35.003012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e61f51-8131-4263-9d57-cbccee3e1c14 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:35.107741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e61f51-8131-4263-9d57-cbccee3e1c14 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:35.143533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b23e5-5b10-42e1-bae6-bc9109e019f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:35.144196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b23e5-5b10-42e1-bae6-bc9109e019f5 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:35.144470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b23e5-5b10-42e1-bae6-bc9109e019f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:35.144738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b23e5-5b10-42e1-bae6-bc9109e019f5 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:35.145408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b23e5-5b10-42e1-bae6-bc9109e019f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:35.146793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b23e5-5b10-42e1-bae6-bc9109e019f5 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:35.158980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 495/989] 10.222.0.79 () {56 vars in 1087 bytes} [Fri May 13 13:42:35 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:42:36.271422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cc773-5ed4-4555-b45b-6bed1aa74345 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:36.272273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cc773-5ed4-4555-b45b-6bed1aa74345 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:36.272273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cc773-5ed4-4555-b45b-6bed1aa74345 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:36.273786 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 495/990] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:36.284185 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c28af-23f3-4e03-9246-cb51b32cd541 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:36.284881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c28af-23f3-4e03-9246-cb51b32cd541 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:36.285157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c28af-23f3-4e03-9246-cb51b32cd541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:36.313903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-124c28af-23f3-4e03-9246-cb51b32cd541 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:36.316381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-124c28af-23f3-4e03-9246-cb51b32cd541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:36.435958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 496/991] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:36.462005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cc773-5ed4-4555-b45b-6bed1aa74345 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:36.604511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cc773-5ed4-4555-b45b-6bed1aa74345 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:36.640742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a9aa2-853b-4909-a234-43cd1971ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:36.641606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9aa2-853b-4909-a234-43cd1971ac68 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:36.641881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9aa2-853b-4909-a234-43cd1971ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:36.642149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9aa2-853b-4909-a234-43cd1971ac68 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:36.643419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a9aa2-853b-4909-a234-43cd1971ac68 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:36.645302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a9aa2-853b-4909-a234-43cd1971ac68 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-593a9aa2-853b-4909-a234-43cd1971ac68 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:36.659454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:36.660645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:36.666748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 496/992] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:36.669193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c28af-23f3-4e03-9246-cb51b32cd541 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:36.788034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c28af-23f3-4e03-9246-cb51b32cd541 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:36.853897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031b41b-3f01-4e28-b89d-f36ac1fee249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:36.854765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031b41b-3f01-4e28-b89d-f36ac1fee249 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:36.855114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031b41b-3f01-4e28-b89d-f36ac1fee249 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:36.855388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031b41b-3f01-4e28-b89d-f36ac1fee249 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:36.856237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031b41b-3f01-4e28-b89d-f36ac1fee249 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:36.856545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031b41b-3f01-4e28-b89d-f36ac1fee249 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:36.858170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031b41b-3f01-4e28-b89d-f36ac1fee249 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:36.874245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 497/993] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:36.876528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a9aa2-853b-4909-a234-43cd1971ac68 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:36.992163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88538bd4-7871-4d5a-8861-a3a17a6abb80 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:36.992490 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88538bd4-7871-4d5a-8861-a3a17a6abb80 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:36.992727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88538bd4-7871-4d5a-8861-a3a17a6abb80 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:36.994361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 498/994] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:36.999822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c924da2-b158-41a0-b5d0-a963b339a332 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:37.000123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c924da2-b158-41a0-b5d0-a963b339a332 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:37.000409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c924da2-b158-41a0-b5d0-a963b339a332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:37.021116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-4c924da2-b158-41a0-b5d0-a963b339a332 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:37.022309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-4c924da2-b158-41a0-b5d0-a963b339a332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:37.025587 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a9aa2-853b-4909-a234-43cd1971ac68 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:37.064530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8fe6f-4d82-4364-a663-a7e60e3db7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:37.065263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8fe6f-4d82-4364-a663-a7e60e3db7b9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:37.065541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8fe6f-4d82-4364-a663-a7e60e3db7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:37.065871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8fe6f-4d82-4364-a663-a7e60e3db7b9 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:37.066600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8fe6f-4d82-4364-a663-a7e60e3db7b9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:37.066927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8fe6f-4d82-4364-a663-a7e60e3db7b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:37.068309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8fe6f-4d82-4364-a663-a7e60e3db7b9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:37.078612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 497/995] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:37.080016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8fe6f-4d82-4364-a663-a7e60e3db7b9 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:37.100528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 499/996] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:37.123059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c924da2-b158-41a0-b5d0-a963b339a332 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:37.232239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8fe6f-4d82-4364-a663-a7e60e3db7b9 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:37.254878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c924da2-b158-41a0-b5d0-a963b339a332 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:37.282096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1bc96-1601-4060-a400-9c863d23f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:37.282799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1bc96-1601-4060-a400-9c863d23f7cb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:37.283682 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1bc96-1601-4060-a400-9c863d23f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:37.283976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1bc96-1601-4060-a400-9c863d23f7cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:37.284769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1bc96-1601-4060-a400-9c863d23f7cb None admin] RBAC: Adding query filter params (interface=public, service_id=249a79dc911f4dfdbcc3ad61b95e98ca, region_id=RegionOne) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:37.285083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1bc96-1601-4060-a400-9c863d23f7cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:37.286473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1bc96-1601-4060-a400-9c863d23f7cb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:37.292113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 498/997] 10.222.0.79 () {56 vars in 1251 bytes} [Fri May 13 13:42:37 2022] GET /identity/v3/endpoints?service_id=249a79dc911f4dfdbcc3ad61b95e98ca&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:37.297794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1bc96-1601-4060-a400-9c863d23f7cb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:37.306532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43357148-7e7a-4142-b2e2-6684b665baaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:37.308010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43357148-7e7a-4142-b2e2-6684b665baaa None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:37.308322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43357148-7e7a-4142-b2e2-6684b665baaa None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:37.308554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43357148-7e7a-4142-b2e2-6684b665baaa None admin] PATH_INFO: `/v3/services/image` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:37.309209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43357148-7e7a-4142-b2e2-6684b665baaa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:37.310596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43357148-7e7a-4142-b2e2-6684b665baaa None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-43357148-7e7a-4142-b2e2-6684b665baaa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:37.324500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:37.325748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:37.328180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 500/998] 10.222.0.79 () {56 vars in 1105 bytes} [Fri May 13 13:42:37 2022] GET /identity/v3/services/image => generated 86 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:37.335032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43357148-7e7a-4142-b2e2-6684b665baaa None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:37.433356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1bc96-1601-4060-a400-9c863d23f7cb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:37.471477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7bcf8-ce7a-49ba-be28-d88926c3ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:37.472146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7bcf8-ce7a-49ba-be28-d88926c3ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:37.472446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7bcf8-ce7a-49ba-be28-d88926c3ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:37.472706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7bcf8-ce7a-49ba-be28-d88926c3ecb2 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:37.473467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7bcf8-ce7a-49ba-be28-d88926c3ecb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:37.475660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7bcf8-ce7a-49ba-be28-d88926c3ecb2 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:37.485746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 499/999] 10.222.0.79 () {56 vars in 1087 bytes} [Fri May 13 13:42:37 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:42:37.489659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43357148-7e7a-4142-b2e2-6684b665baaa None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:37.542768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69941ee9-9df5-4747-b10f-d038ad003ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:37.543451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69941ee9-9df5-4747-b10f-d038ad003ebb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:37.543728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69941ee9-9df5-4747-b10f-d038ad003ebb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:37.543994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69941ee9-9df5-4747-b10f-d038ad003ebb None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:37.544734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69941ee9-9df5-4747-b10f-d038ad003ebb None admin] RBAC: Adding query filter params (name=image) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:37.545037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69941ee9-9df5-4747-b10f-d038ad003ebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:37.546639 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69941ee9-9df5-4747-b10f-d038ad003ebb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:37.552887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 501/1000] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:37.560256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7bcf8-ce7a-49ba-be28-d88926c3ecb2 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:37.691359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609e112-db59-4676-bb4b-458954d3e446 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:37.691538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609e112-db59-4676-bb4b-458954d3e446 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:37.691816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609e112-db59-4676-bb4b-458954d3e446 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:37.693176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 502/1001] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:37.699147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a5864-10df-46db-9505-aafa1aa192ce None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:37.699452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a5864-10df-46db-9505-aafa1aa192ce None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:37.699704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a5864-10df-46db-9505-aafa1aa192ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:37.708398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7bcf8-ce7a-49ba-be28-d88926c3ecb2 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:37.721067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1d6a5864-10df-46db-9505-aafa1aa192ce None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:37.722269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1d6a5864-10df-46db-9505-aafa1aa192ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:37.743742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb1500-4526-4d3a-9835-1e222cc1ded8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:37.744380 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb1500-4526-4d3a-9835-1e222cc1ded8 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:37.744658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb1500-4526-4d3a-9835-1e222cc1ded8 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:37.744930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb1500-4526-4d3a-9835-1e222cc1ded8 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:37.745633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb1500-4526-4d3a-9835-1e222cc1ded8 None admin] RBAC: Adding query filter params (type=image) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:37.745939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb1500-4526-4d3a-9835-1e222cc1ded8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:37.747367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb1500-4526-4d3a-9835-1e222cc1ded8 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:37.753352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 500/1002] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:37.795298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 503/1003] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:37.818590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb1500-4526-4d3a-9835-1e222cc1ded8 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:37.934353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb1500-4526-4d3a-9835-1e222cc1ded8 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:37.980363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4d67b-a568-4fcb-8c63-3ebd3a3cf021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:37.981212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4d67b-a568-4fcb-8c63-3ebd3a3cf021 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:37.981489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4d67b-a568-4fcb-8c63-3ebd3a3cf021 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:37.981767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4d67b-a568-4fcb-8c63-3ebd3a3cf021 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:37.982404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4d67b-a568-4fcb-8c63-3ebd3a3cf021 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:37.983826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4d67b-a568-4fcb-8c63-3ebd3a3cf021 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-58f4d67b-a568-4fcb-8c63-3ebd3a3cf021 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:37.996537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:37.997614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:37.999280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 501/1004] 10.222.0.79 () {56 vars in 1126 bytes} [Fri May 13 13:42:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:38.004243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a5864-10df-46db-9505-aafa1aa192ce None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:38.111507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a5864-10df-46db-9505-aafa1aa192ce None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:38.148757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968510a-ba64-44a0-834e-a32a841c12ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:38.149424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968510a-ba64-44a0-834e-a32a841c12ae None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:38.149703 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968510a-ba64-44a0-834e-a32a841c12ae None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:38.149974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968510a-ba64-44a0-834e-a32a841c12ae None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:38.150672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968510a-ba64-44a0-834e-a32a841c12ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:38.151000 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968510a-ba64-44a0-834e-a32a841c12ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:38.152359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968510a-ba64-44a0-834e-a32a841c12ae None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:38.157154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 504/1005] 10.222.0.79 () {56 vars in 1122 bytes} [Fri May 13 13:42:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:38.163913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4d67b-a568-4fcb-8c63-3ebd3a3cf021 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:38.266823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4d67b-a568-4fcb-8c63-3ebd3a3cf021 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:38.308807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c14ba-7cd8-4b16-853f-437ba74f7e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:38.309735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c14ba-7cd8-4b16-853f-437ba74f7e91 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:38.309909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c14ba-7cd8-4b16-853f-437ba74f7e91 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:38.310186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c14ba-7cd8-4b16-853f-437ba74f7e91 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:38.311115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c14ba-7cd8-4b16-853f-437ba74f7e91 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:38.311427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c14ba-7cd8-4b16-853f-437ba74f7e91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:38.312817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c14ba-7cd8-4b16-853f-437ba74f7e91 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:38.319112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 502/1006] 10.222.0.79 () {56 vars in 1122 bytes} [Fri May 13 13:42:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:38.324883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968510a-ba64-44a0-834e-a32a841c12ae None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:38.431395 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968510a-ba64-44a0-834e-a32a841c12ae None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:38.466675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8c115-86fb-4fed-9b55-ec703a9f6bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:38.467374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8c115-86fb-4fed-9b55-ec703a9f6bb8 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:38.467647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8c115-86fb-4fed-9b55-ec703a9f6bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:38.467921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8c115-86fb-4fed-9b55-ec703a9f6bb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:38.468671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8c115-86fb-4fed-9b55-ec703a9f6bb8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:38.470049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8c115-86fb-4fed-9b55-ec703a9f6bb8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:38.488166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 505/1007] 10.222.0.79 () {60 vars in 1144 bytes} [Fri May 13 13:42:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:42:39.880167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68813e-5b4b-44b7-a554-7cc707fb34ef None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:39.881173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68813e-5b4b-44b7-a554-7cc707fb34ef None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:39.881173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68813e-5b4b-44b7-a554-7cc707fb34ef None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:39.882672 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 503/1008] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:39.888804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5d933-1f64-4552-883c-6a8cc5a5c497 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:39.889468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5d933-1f64-4552-883c-6a8cc5a5c497 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:39.889468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5d933-1f64-4552-883c-6a8cc5a5c497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:39.910957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-12a5d933-1f64-4552-883c-6a8cc5a5c497 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:39.910957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-12a5d933-1f64-4552-883c-6a8cc5a5c497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:40.010759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 506/1009] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:40.034757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68813e-5b4b-44b7-a554-7cc707fb34ef None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:40.157694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68813e-5b4b-44b7-a554-7cc707fb34ef None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:40.195375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5f267-541e-4b8b-88dd-5d152f85c7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:40.196576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5f267-541e-4b8b-88dd-5d152f85c7fa None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:40.197085 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5f267-541e-4b8b-88dd-5d152f85c7fa None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:40.197574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5f267-541e-4b8b-88dd-5d152f85c7fa None admin] PATH_INFO: `/v3/services/compute` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:40.198471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5f267-541e-4b8b-88dd-5d152f85c7fa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:40.200365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5f267-541e-4b8b-88dd-5d152f85c7fa None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-e9a5f267-541e-4b8b-88dd-5d152f85c7fa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:40.213705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:40.215082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:42:40.216055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:40.218771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e8460-5336-4886-86aa-ce1f6e6f3f56 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:40.219274 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e8460-5336-4886-86aa-ce1f6e6f3f56 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:40.219558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e8460-5336-4886-86aa-ce1f6e6f3f56 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:40.221107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 504/1010] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:40.221513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 507/1011] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:40.222317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5f267-541e-4b8b-88dd-5d152f85c7fa None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:40.228971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6902018-fe09-494e-91f2-e98646041c05 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:40.229320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6902018-fe09-494e-91f2-e98646041c05 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:40.229514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6902018-fe09-494e-91f2-e98646041c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:40.248454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b6902018-fe09-494e-91f2-e98646041c05 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:40.249657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b6902018-fe09-494e-91f2-e98646041c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:40.343208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 508/1012] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:40.361914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5f267-541e-4b8b-88dd-5d152f85c7fa None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:40.370185 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6902018-fe09-494e-91f2-e98646041c05 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:40.402161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1cd958-f467-4d7e-bcf4-5276bb646055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:40.402875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cd958-f467-4d7e-bcf4-5276bb646055 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:40.403211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cd958-f467-4d7e-bcf4-5276bb646055 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:40.403506 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cd958-f467-4d7e-bcf4-5276bb646055 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:40.404280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1cd958-f467-4d7e-bcf4-5276bb646055 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:40.404583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1cd958-f467-4d7e-bcf4-5276bb646055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:40.406177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1cd958-f467-4d7e-bcf4-5276bb646055 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:40.412512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 505/1013] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:40.418902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1cd958-f467-4d7e-bcf4-5276bb646055 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:40.498474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6902018-fe09-494e-91f2-e98646041c05 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:40.549423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee671b07-553d-46ea-9db6-2625cff9b3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:40.550523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee671b07-553d-46ea-9db6-2625cff9b3e3 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:40.551019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee671b07-553d-46ea-9db6-2625cff9b3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:40.551481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee671b07-553d-46ea-9db6-2625cff9b3e3 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:40.552731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee671b07-553d-46ea-9db6-2625cff9b3e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:40.555251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee671b07-553d-46ea-9db6-2625cff9b3e3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:40.586745 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 509/1014] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:42:40 2022] POST /identity/v3/services => generated 242 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:42:40.587036 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1cd958-f467-4d7e-bcf4-5276bb646055 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:40.648743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b9567-034b-45e0-ba5c-f43196d20070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:40.649512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b9567-034b-45e0-ba5c-f43196d20070 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:40.649899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b9567-034b-45e0-ba5c-f43196d20070 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:40.650264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b9567-034b-45e0-ba5c-f43196d20070 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:40.651393 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b9567-034b-45e0-ba5c-f43196d20070 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:40.651750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b9567-034b-45e0-ba5c-f43196d20070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:40.653309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b9567-034b-45e0-ba5c-f43196d20070 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:40.661475 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 506/1015] 10.222.0.79 () {56 vars in 1112 bytes} [Fri May 13 13:42:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:40.668091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee671b07-553d-46ea-9db6-2625cff9b3e3 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:40.824710 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee671b07-553d-46ea-9db6-2625cff9b3e3 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:40.863372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2af74d-4c74-42f4-a466-ad4f02c19b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:40.864398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2af74d-4c74-42f4-a466-ad4f02c19b1c None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:40.864844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2af74d-4c74-42f4-a466-ad4f02c19b1c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:40.865295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2af74d-4c74-42f4-a466-ad4f02c19b1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:40.866201 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2af74d-4c74-42f4-a466-ad4f02c19b1c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:40.867775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2af74d-4c74-42f4-a466-ad4f02c19b1c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:40.888241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392bf65-aa51-488d-a722-14a432a8ecaf None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:40.888793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392bf65-aa51-488d-a722-14a432a8ecaf None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:40.888793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392bf65-aa51-488d-a722-14a432a8ecaf None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:40.890292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 507/1016] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:40.896131 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fbeb9-522f-46e0-847c-2815b9f928f0 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:40.896508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fbeb9-522f-46e0-847c-2815b9f928f0 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:40.896627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fbeb9-522f-46e0-847c-2815b9f928f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:40.907219 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 510/1017] 10.222.0.79 () {60 vars in 1144 bytes} [Fri May 13 13:42:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:42:40.920154 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-858fbeb9-522f-46e0-847c-2815b9f928f0 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:40.921817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-858fbeb9-522f-46e0-847c-2815b9f928f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:41.014832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 508/1018] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:41.036568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2af74d-4c74-42f4-a466-ad4f02c19b1c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:41.193533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2af74d-4c74-42f4-a466-ad4f02c19b1c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:41.242337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76317-e273-458b-a812-0c61d522c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:41.243199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76317-e273-458b-a812-0c61d522c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:41.243523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76317-e273-458b-a812-0c61d522c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:41.243750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76317-e273-458b-a812-0c61d522c7f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:41.244986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b76317-e273-458b-a812-0c61d522c7f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:41.246452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b76317-e273-458b-a812-0c61d522c7f3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:41.248495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 511/1019] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:41.255345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fbeb9-522f-46e0-847c-2815b9f928f0 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:41.376102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fbeb9-522f-46e0-847c-2815b9f928f0 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:41.417284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be0f37-6a70-44c8-8a9f-0fd47e301bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:41.417976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be0f37-6a70-44c8-8a9f-0fd47e301bd4 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:41.418256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be0f37-6a70-44c8-8a9f-0fd47e301bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:41.418531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be0f37-6a70-44c8-8a9f-0fd47e301bd4 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:41.419303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be0f37-6a70-44c8-8a9f-0fd47e301bd4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:41.420771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be0f37-6a70-44c8-8a9f-0fd47e301bd4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:41.449867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 509/1020] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:42:41 2022] POST /identity/v3/projects => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:42:43.105105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494bdac-9594-4024-a456-4338973a4256 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:43.105668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494bdac-9594-4024-a456-4338973a4256 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:43.105668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494bdac-9594-4024-a456-4338973a4256 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:43.107752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 512/1021] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:43.113676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf784d1-c7ee-4504-a5a3-39db6c386e64 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:43.114255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf784d1-c7ee-4504-a5a3-39db6c386e64 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:43.114255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf784d1-c7ee-4504-a5a3-39db6c386e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:43.131993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-adf784d1-c7ee-4504-a5a3-39db6c386e64 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:43.133203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-adf784d1-c7ee-4504-a5a3-39db6c386e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:43.207113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 510/1022] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:43.229873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494bdac-9594-4024-a456-4338973a4256 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:43.351336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494bdac-9594-4024-a456-4338973a4256 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:43.398018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6bbe5-d4a0-4acf-9bbf-33bd44ae4886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:43.398873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6bbe5-d4a0-4acf-9bbf-33bd44ae4886 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:43.399159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6bbe5-d4a0-4acf-9bbf-33bd44ae4886 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:43.399435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6bbe5-d4a0-4acf-9bbf-33bd44ae4886 None admin] PATH_INFO: `/v3/services/image` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:43.400223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6bbe5-d4a0-4acf-9bbf-33bd44ae4886 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:43.401668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6bbe5-d4a0-4acf-9bbf-33bd44ae4886 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-87d6bbe5-d4a0-4acf-9bbf-33bd44ae4886 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:43.418473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:43.426935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:43.428487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 513/1023] 10.222.0.79 () {56 vars in 1105 bytes} [Fri May 13 13:42:43 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:43.434971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf784d1-c7ee-4504-a5a3-39db6c386e64 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:43.472913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe0078-e85b-4150-b31b-bf929c7700e5 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:43.473218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe0078-e85b-4150-b31b-bf929c7700e5 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:43.473456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe0078-e85b-4150-b31b-bf929c7700e5 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:43.475180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 514/1024] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:43.481828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc348-4b18-4cbd-9618-7ff91d838e14 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:43.481828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc348-4b18-4cbd-9618-7ff91d838e14 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:43.482181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbc348-4b18-4cbd-9618-7ff91d838e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:43.507112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-eddbc348-4b18-4cbd-9618-7ff91d838e14 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:43.508940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-eddbc348-4b18-4cbd-9618-7ff91d838e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:43.595786 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf784d1-c7ee-4504-a5a3-39db6c386e64 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:43.624001 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 515/1025] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:43.650596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c083e2e-03af-46fc-96fb-17ccecef167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:43.652279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c083e2e-03af-46fc-96fb-17ccecef167a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:43.653014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c083e2e-03af-46fc-96fb-17ccecef167a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:43.653707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c083e2e-03af-46fc-96fb-17ccecef167a None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:43.655317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c083e2e-03af-46fc-96fb-17ccecef167a None admin] RBAC: Adding query filter params (name=image) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:43.656093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c083e2e-03af-46fc-96fb-17ccecef167a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:43.658477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbc348-4b18-4cbd-9618-7ff91d838e14 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:43.659475 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c083e2e-03af-46fc-96fb-17ccecef167a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:43.670486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 511/1026] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:43.678507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c083e2e-03af-46fc-96fb-17ccecef167a None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:43.847652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbc348-4b18-4cbd-9618-7ff91d838e14 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:43.857115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c083e2e-03af-46fc-96fb-17ccecef167a None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:43.897473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be4ae8-8864-459f-974b-02083d08d1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:43.898920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be4ae8-8864-459f-974b-02083d08d1a7 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:43.899371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be4ae8-8864-459f-974b-02083d08d1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:43.899806 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be4ae8-8864-459f-974b-02083d08d1a7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:43.920145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be4ae8-8864-459f-974b-02083d08d1a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-57be4ae8-8864-459f-974b-02083d08d1a7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:43.930073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:43.931293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:43.932316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:43.936776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 516/1027] 10.222.0.79 () {56 vars in 1120 bytes} [Fri May 13 13:42:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:43.937746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be4ae8-8864-459f-974b-02083d08d1a7 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:43.938075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3280d9-74e6-4dc7-baa9-0ebdad69abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:43.938875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3280d9-74e6-4dc7-baa9-0ebdad69abfd None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:43.939745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3280d9-74e6-4dc7-baa9-0ebdad69abfd None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:43.940053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3280d9-74e6-4dc7-baa9-0ebdad69abfd None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:43.940802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3280d9-74e6-4dc7-baa9-0ebdad69abfd None admin] RBAC: Adding query filter params (type=image) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:43.941308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3280d9-74e6-4dc7-baa9-0ebdad69abfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:43.942736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3280d9-74e6-4dc7-baa9-0ebdad69abfd None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:43.949120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 512/1028] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:43.955259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3280d9-74e6-4dc7-baa9-0ebdad69abfd None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:44.078484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be4ae8-8864-459f-974b-02083d08d1a7 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:44.089486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3280d9-74e6-4dc7-baa9-0ebdad69abfd None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:44.119186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a60403-ccf5-4380-8c7c-06046493c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:44.119922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a60403-ccf5-4380-8c7c-06046493c226 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:44.120078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a60403-ccf5-4380-8c7c-06046493c226 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:44.120335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a60403-ccf5-4380-8c7c-06046493c226 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:44.121087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a60403-ccf5-4380-8c7c-06046493c226 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:44.121380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a60403-ccf5-4380-8c7c-06046493c226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:44.122805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a60403-ccf5-4380-8c7c-06046493c226 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:44.135615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 517/1029] 10.222.0.79 () {56 vars in 1115 bytes} [Fri May 13 13:42:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:44.138542 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b7834-aa56-4337-8c42-d8ab8cac827d None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:44.139148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b7834-aa56-4337-8c42-d8ab8cac827d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:44.139465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b7834-aa56-4337-8c42-d8ab8cac827d None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:44.141686 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 518/1030] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:44.147350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b7834-aa56-4337-8c42-d8ab8cac827d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:44.148851 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d166b-66c9-45c9-a426-4225efb1a827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:44.149990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d166b-66c9-45c9-a426-4225efb1a827 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:44.150498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d166b-66c9-45c9-a426-4225efb1a827 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:44.151001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d166b-66c9-45c9-a426-4225efb1a827 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:44.152523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d166b-66c9-45c9-a426-4225efb1a827 None admin] RBAC: Adding query filter params (interface=public, service_id=c3c2e0d5cd384d8fb16a8a64b6357b92, region_id=RegionOne) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:44.152849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d166b-66c9-45c9-a426-4225efb1a827 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:44.155394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d166b-66c9-45c9-a426-4225efb1a827 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:44.165135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 513/1031] 10.222.0.79 () {56 vars in 1251 bytes} [Fri May 13 13:42:43 2022] GET /identity/v3/endpoints?service_id=c3c2e0d5cd384d8fb16a8a64b6357b92&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:44.167960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5139d3-b696-4b18-b50a-8c1195e91acf None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:44.168441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5139d3-b696-4b18-b50a-8c1195e91acf None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:44.168925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5139d3-b696-4b18-b50a-8c1195e91acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:44.204263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-4c5139d3-b696-4b18-b50a-8c1195e91acf None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:44.206233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4c5139d3-b696-4b18-b50a-8c1195e91acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:44.296690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 514/1032] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:44.297835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5139d3-b696-4b18-b50a-8c1195e91acf None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:44.315139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b7834-aa56-4337-8c42-d8ab8cac827d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:44.367017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f743b1-546f-4ebb-9fbf-f88cada25234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:44.368091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f743b1-546f-4ebb-9fbf-f88cada25234 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:44.368511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f743b1-546f-4ebb-9fbf-f88cada25234 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:44.368819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f743b1-546f-4ebb-9fbf-f88cada25234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:44.385947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f743b1-546f-4ebb-9fbf-f88cada25234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:44.387373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f743b1-546f-4ebb-9fbf-f88cada25234 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-a1f743b1-546f-4ebb-9fbf-f88cada25234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:44.406018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:44.407442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:44.408837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:44.408837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:44.408837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:44.408837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:44.408837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 519/1033] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:44.408837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f743b1-546f-4ebb-9fbf-f88cada25234 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:44.437543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5139d3-b696-4b18-b50a-8c1195e91acf None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:44.474005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b882cf-e081-45ee-9186-7d5304f99de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:44.474665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b882cf-e081-45ee-9186-7d5304f99de7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:44.474968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b882cf-e081-45ee-9186-7d5304f99de7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:44.475238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b882cf-e081-45ee-9186-7d5304f99de7 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:44.475916 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b882cf-e081-45ee-9186-7d5304f99de7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:44.477283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b882cf-e081-45ee-9186-7d5304f99de7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:44.488768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 515/1034] 10.222.0.79 () {56 vars in 1087 bytes} [Fri May 13 13:42:44 2022] GET /identity/v3/services => generated 1957 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:42:44.490206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b882cf-e081-45ee-9186-7d5304f99de7 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:44.543848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f743b1-546f-4ebb-9fbf-f88cada25234 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:44.592600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db0be9-8168-4a3e-ad48-7f20fd98ada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:44.593428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db0be9-8168-4a3e-ad48-7f20fd98ada4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:44.593708 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db0be9-8168-4a3e-ad48-7f20fd98ada4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:44.593985 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db0be9-8168-4a3e-ad48-7f20fd98ada4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:44.595426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db0be9-8168-4a3e-ad48-7f20fd98ada4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:44.596959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db0be9-8168-4a3e-ad48-7f20fd98ada4 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:44.598912 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 520/1035] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:44.605000 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db0be9-8168-4a3e-ad48-7f20fd98ada4 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:44.628888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b882cf-e081-45ee-9186-7d5304f99de7 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:44.680538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de7510-ef77-4c75-b17a-ff1bf780f9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:44.681520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de7510-ef77-4c75-b17a-ff1bf780f9a0 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:44.681981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de7510-ef77-4c75-b17a-ff1bf780f9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:44.682453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de7510-ef77-4c75-b17a-ff1bf780f9a0 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:44.684151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de7510-ef77-4c75-b17a-ff1bf780f9a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:44.684628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de7510-ef77-4c75-b17a-ff1bf780f9a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:44.687269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de7510-ef77-4c75-b17a-ff1bf780f9a0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:44.720110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 516/1036] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:44.726835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de7510-ef77-4c75-b17a-ff1bf780f9a0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:44.766190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db0be9-8168-4a3e-ad48-7f20fd98ada4 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:44.820720 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b3f63-31f4-40cf-aee0-99262656355f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:44.821972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b3f63-31f4-40cf-aee0-99262656355f None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:44.822440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b3f63-31f4-40cf-aee0-99262656355f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:44.822911 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b3f63-31f4-40cf-aee0-99262656355f None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:44.824302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b3f63-31f4-40cf-aee0-99262656355f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:44.827456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b3f63-31f4-40cf-aee0-99262656355f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:44.858240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 521/1037] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:42:44 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:42:44.907298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de7510-ef77-4c75-b17a-ff1bf780f9a0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:44.975776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad476aa-28a1-412c-9746-e7545827489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:44.978850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad476aa-28a1-412c-9746-e7545827489b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:44.979651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad476aa-28a1-412c-9746-e7545827489b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:44.980504 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad476aa-28a1-412c-9746-e7545827489b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:44.995677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad476aa-28a1-412c-9746-e7545827489b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:44.999083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad476aa-28a1-412c-9746-e7545827489b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-bad476aa-28a1-412c-9746-e7545827489b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:45.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:45.031454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:45.033827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:45.033827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:45.033827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:45.033827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:45.033827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:45.033827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:45.033827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:45.033827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:45.033827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:42:45.033827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:45.037577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 517/1038] 10.222.0.79 () {56 vars in 1093 bytes} [Fri May 13 13:42:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:45.038734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b3f63-31f4-40cf-aee0-99262656355f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:45.194094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b3f63-31f4-40cf-aee0-99262656355f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:45.249038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092668b-6bef-442c-925f-dcd9974339f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:45.249623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092668b-6bef-442c-925f-dcd9974339f3 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:45.249895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092668b-6bef-442c-925f-dcd9974339f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:45.250161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092668b-6bef-442c-925f-dcd9974339f3 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:45.251025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092668b-6bef-442c-925f-dcd9974339f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:45.251899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092668b-6bef-442c-925f-dcd9974339f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:45.253446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092668b-6bef-442c-925f-dcd9974339f3 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:45.274007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 522/1039] 10.222.0.79 () {56 vars in 1133 bytes} [Fri May 13 13:42:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:45.280686 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad476aa-28a1-412c-9746-e7545827489b None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:45.440193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad476aa-28a1-412c-9746-e7545827489b None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:45.505886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a22d8-ef57-434a-9900-6068e4f3a869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:45.506818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a22d8-ef57-434a-9900-6068e4f3a869 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:45.507291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a22d8-ef57-434a-9900-6068e4f3a869 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:45.507736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a22d8-ef57-434a-9900-6068e4f3a869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:45.637789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a22d8-ef57-434a-9900-6068e4f3a869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:45.640441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a22d8-ef57-434a-9900-6068e4f3a869 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:45.793618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 518/1040] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:42:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:42:45.800630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092668b-6bef-442c-925f-dcd9974339f3 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:45.943275 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092668b-6bef-442c-925f-dcd9974339f3 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:45.989968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a0045-45fe-4f56-aba8-9f46f8783b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:45.990815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a0045-45fe-4f56-aba8-9f46f8783b63 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:45.991090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a0045-45fe-4f56-aba8-9f46f8783b63 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:45.991361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a0045-45fe-4f56-aba8-9f46f8783b63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:46.002225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a0045-45fe-4f56-aba8-9f46f8783b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:46.003725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a0045-45fe-4f56-aba8-9f46f8783b63 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-756a0045-45fe-4f56-aba8-9f46f8783b63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:46.023024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:46.024091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:46.025009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:46.025009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:46.025009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:46.025009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:46.027041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 523/1041] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:46.030361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a22d8-ef57-434a-9900-6068e4f3a869 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:46.141551 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a22d8-ef57-434a-9900-6068e4f3a869 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:46.176651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1100f02-2567-4396-9027-c05d7a005051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:46.177320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1100f02-2567-4396-9027-c05d7a005051 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:46.177600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1100f02-2567-4396-9027-c05d7a005051 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:46.177878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1100f02-2567-4396-9027-c05d7a005051 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:46.178599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1100f02-2567-4396-9027-c05d7a005051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:46.178939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1100f02-2567-4396-9027-c05d7a005051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:46.180313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1100f02-2567-4396-9027-c05d7a005051 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:46.191996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 519/1042] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:46.197367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a0045-45fe-4f56-aba8-9f46f8783b63 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:46.349209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a0045-45fe-4f56-aba8-9f46f8783b63 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:46.394414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e82abe-3f6a-42e5-8fa5-ecadc8c419e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:46.395495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e82abe-3f6a-42e5-8fa5-ecadc8c419e8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:46.395952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e82abe-3f6a-42e5-8fa5-ecadc8c419e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:46.396372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e82abe-3f6a-42e5-8fa5-ecadc8c419e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:46.404367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e82abe-3f6a-42e5-8fa5-ecadc8c419e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:46.406072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e82abe-3f6a-42e5-8fa5-ecadc8c419e8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-94e82abe-3f6a-42e5-8fa5-ecadc8c419e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:46.421469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:46.422557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:46.423500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:46.423500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:46.423500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:46.423500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:46.423500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:46.423500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:46.423500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:46.423500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:46.424336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 524/1043] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:46.429414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1100f02-2567-4396-9027-c05d7a005051 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:46.550930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1100f02-2567-4396-9027-c05d7a005051 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:46.604114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138491a6-aa82-4f00-af1b-c48044efc36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:46.604788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138491a6-aa82-4f00-af1b-c48044efc36b None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:46.605073 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138491a6-aa82-4f00-af1b-c48044efc36b None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:46.605347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138491a6-aa82-4f00-af1b-c48044efc36b None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:46.606134 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138491a6-aa82-4f00-af1b-c48044efc36b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:46.606440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138491a6-aa82-4f00-af1b-c48044efc36b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:46.608038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138491a6-aa82-4f00-af1b-c48044efc36b None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:46.621609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 520/1044] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:46.628256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e82abe-3f6a-42e5-8fa5-ecadc8c419e8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:46.784988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e82abe-3f6a-42e5-8fa5-ecadc8c419e8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:46.821569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fadbfe-0155-440b-b8bf-987eac8afb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:46.822227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fadbfe-0155-440b-b8bf-987eac8afb6d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:46.822507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fadbfe-0155-440b-b8bf-987eac8afb6d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:46.822812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fadbfe-0155-440b-b8bf-987eac8afb6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:46.823919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fadbfe-0155-440b-b8bf-987eac8afb6d None admin] RBAC: Adding query filter params (role.id=f4a90110e2c845408f5ec7685f7ef2d0, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=24ad0dd9d265425087be2be8d245b7fc) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:46.824248 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fadbfe-0155-440b-b8bf-987eac8afb6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:46.825668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fadbfe-0155-440b-b8bf-987eac8afb6d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:46.833362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 525/1045] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:42:46 2022] GET /identity/v3/role_assignments?user.id=24ad0dd9d265425087be2be8d245b7fc&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=f4a90110e2c845408f5ec7685f7ef2d0 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:47.051196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b70015-6760-4c07-aab1-531c4e80c136 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:47.051196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b70015-6760-4c07-aab1-531c4e80c136 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:47.051196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b70015-6760-4c07-aab1-531c4e80c136 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:47.052788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 521/1046] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:47.063296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b75df-924b-41e7-b7e3-c3762a2d3626 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:47.063605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b75df-924b-41e7-b7e3-c3762a2d3626 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:47.064409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b75df-924b-41e7-b7e3-c3762a2d3626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:47.083704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-379b75df-924b-41e7-b7e3-c3762a2d3626 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:47.084906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-379b75df-924b-41e7-b7e3-c3762a2d3626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:47.171092 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 526/1047] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:47.194164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b70015-6760-4c07-aab1-531c4e80c136 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:47.348771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b70015-6760-4c07-aab1-531c4e80c136 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:47.398888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f084f-9c94-41c9-a6e4-f7d2e08667ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:47.399913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f084f-9c94-41c9-a6e4-f7d2e08667ca None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:47.400258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f084f-9c94-41c9-a6e4-f7d2e08667ca None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:47.400623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f084f-9c94-41c9-a6e4-f7d2e08667ca None admin] PATH_INFO: `/v3/services/image` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:47.401493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f084f-9c94-41c9-a6e4-f7d2e08667ca None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:47.403434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f084f-9c94-41c9-a6e4-f7d2e08667ca None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-f71f084f-9c94-41c9-a6e4-f7d2e08667ca None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:47.420519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:47.421787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:42:47.425040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:47.426968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 522/1048] 10.222.0.79 () {56 vars in 1105 bytes} [Fri May 13 13:42:47 2022] GET /identity/v3/services/image => generated 86 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:47.430838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b75df-924b-41e7-b7e3-c3762a2d3626 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:47.578417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0ec10-5724-4df4-a36b-255570b2de58 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:47.579226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0ec10-5724-4df4-a36b-255570b2de58 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:47.579226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0ec10-5724-4df4-a36b-255570b2de58 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:47.580489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 523/1049] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:47.586462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33363d9-5d93-4037-9425-3a443bdbd2c5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:47.586860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33363d9-5d93-4037-9425-3a443bdbd2c5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:47.587129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33363d9-5d93-4037-9425-3a443bdbd2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:47.589864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b75df-924b-41e7-b7e3-c3762a2d3626 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:47.610511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c33363d9-5d93-4037-9425-3a443bdbd2c5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:47.612374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c33363d9-5d93-4037-9425-3a443bdbd2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:47.634049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa9545-0276-44ff-a529-3ad31895cd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:47.635144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa9545-0276-44ff-a529-3ad31895cd1c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:47.635593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa9545-0276-44ff-a529-3ad31895cd1c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:47.636040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa9545-0276-44ff-a529-3ad31895cd1c None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:47.637251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa9545-0276-44ff-a529-3ad31895cd1c None admin] RBAC: Adding query filter params (name=image) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:47.637767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa9545-0276-44ff-a529-3ad31895cd1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:47.640723 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa9545-0276-44ff-a529-3ad31895cd1c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:47.654424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 527/1050] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:47.655784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa9545-0276-44ff-a529-3ad31895cd1c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:47.717459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 524/1051] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:47.743428 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33363d9-5d93-4037-9425-3a443bdbd2c5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:47.778152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa9545-0276-44ff-a529-3ad31895cd1c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:47.820237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06589f7-938d-4b84-83b6-c4464b36ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:47.820237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06589f7-938d-4b84-83b6-c4464b36ac6f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:47.820237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06589f7-938d-4b84-83b6-c4464b36ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:47.820237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06589f7-938d-4b84-83b6-c4464b36ac6f None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:47.820844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06589f7-938d-4b84-83b6-c4464b36ac6f None admin] RBAC: Adding query filter params (type=image) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:47.820844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06589f7-938d-4b84-83b6-c4464b36ac6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:47.822203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06589f7-938d-4b84-83b6-c4464b36ac6f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:47.827750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 528/1052] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:47.834039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06589f7-938d-4b84-83b6-c4464b36ac6f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:47.852532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33363d9-5d93-4037-9425-3a443bdbd2c5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:47.904550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f7f48-90b7-4bf8-b2a0-c588b3af6a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:47.905589 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f7f48-90b7-4bf8-b2a0-c588b3af6a1e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:47.906021 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f7f48-90b7-4bf8-b2a0-c588b3af6a1e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:47.906463 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f7f48-90b7-4bf8-b2a0-c588b3af6a1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:47.922440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f7f48-90b7-4bf8-b2a0-c588b3af6a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-a67f7f48-90b7-4bf8-b2a0-c588b3af6a1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:47.931263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:47.932389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:47.938367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 525/1053] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:42:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:47.941283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f7f48-90b7-4bf8-b2a0-c588b3af6a1e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:47.960176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06589f7-938d-4b84-83b6-c4464b36ac6f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:48.011650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df323d5-ee04-434f-9cfc-862b189f120b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:48.012513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df323d5-ee04-434f-9cfc-862b189f120b None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:48.012962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df323d5-ee04-434f-9cfc-862b189f120b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:48.013409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df323d5-ee04-434f-9cfc-862b189f120b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:48.014776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df323d5-ee04-434f-9cfc-862b189f120b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:48.017387 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df323d5-ee04-434f-9cfc-862b189f120b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:48.046946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 529/1054] 10.222.0.79 () {60 vars in 1144 bytes} [Fri May 13 13:42:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:42:48.100492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f7f48-90b7-4bf8-b2a0-c588b3af6a1e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:48.169317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344aa497-036c-4219-b70a-937841354b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:48.170091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344aa497-036c-4219-b70a-937841354b22 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:48.170585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344aa497-036c-4219-b70a-937841354b22 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:48.171063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344aa497-036c-4219-b70a-937841354b22 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:48.172497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344aa497-036c-4219-b70a-937841354b22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:48.173685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344aa497-036c-4219-b70a-937841354b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:48.176429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344aa497-036c-4219-b70a-937841354b22 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:48.196427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 526/1055] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:42:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:48.198603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df323d5-ee04-434f-9cfc-862b189f120b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:48.371160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df323d5-ee04-434f-9cfc-862b189f120b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:48.420857 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71288852-2c41-4c6b-a8bd-631c0bce2ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:48.421645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71288852-2c41-4c6b-a8bd-631c0bce2ccc None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:48.421914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71288852-2c41-4c6b-a8bd-631c0bce2ccc None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:48.422179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71288852-2c41-4c6b-a8bd-631c0bce2ccc None admin] PATH_INFO: `/v3/users/2d4b871d08f7414daa5606b8b7d3e0d1` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:48.443825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71288852-2c41-4c6b-a8bd-631c0bce2ccc None admin] RBAC: Authorizing `identity:get_user(user_id=2d4b871d08f7414daa5606b8b7d3e0d1)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:48.446234 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71288852-2c41-4c6b-a8bd-631c0bce2ccc None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:48.448858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 530/1056] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:42:48 2022] GET /identity/v3/users/2d4b871d08f7414daa5606b8b7d3e0d1 => generated 285 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:48.457244 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344aa497-036c-4219-b70a-937841354b22 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:48.627225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344aa497-036c-4219-b70a-937841354b22 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:48.673256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8cb6f-7f78-4595-8282-16df6eb10f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:48.673809 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8cb6f-7f78-4595-8282-16df6eb10f16 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:48.674089 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8cb6f-7f78-4595-8282-16df6eb10f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:48.674364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8cb6f-7f78-4595-8282-16df6eb10f16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:48.751921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8cb6f-7f78-4595-8282-16df6eb10f16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:48.753498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8cb6f-7f78-4595-8282-16df6eb10f16 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:48.918947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 527/1057] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:42:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:42:48.922533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71288852-2c41-4c6b-a8bd-631c0bce2ccc None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:49.092061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71288852-2c41-4c6b-a8bd-631c0bce2ccc None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:49.148073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8b93f-8b08-4de4-9891-167b4d020931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:49.149532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b93f-8b08-4de4-9891-167b4d020931 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:49.150013 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b93f-8b08-4de4-9891-167b4d020931 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:49.150497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b93f-8b08-4de4-9891-167b4d020931 None admin] PATH_INFO: `/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:49.152782 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8b93f-8b08-4de4-9891-167b4d020931 None admin] RBAC: Authorizing `identity:get_project(project_id=4717a71ca9d541939b2f0ba31daf6a0e)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:49.155513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8b93f-8b08-4de4-9891-167b4d020931 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:49.159761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 531/1058] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:42:48 2022] GET /identity/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e => generated 311 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:49.169723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8cb6f-7f78-4595-8282-16df6eb10f16 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:49.340515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8cb6f-7f78-4595-8282-16df6eb10f16 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:49.391765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b5ea1-6c4a-4170-9dbc-0a4b1fdb2e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:49.392624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b5ea1-6c4a-4170-9dbc-0a4b1fdb2e77 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:49.393030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b5ea1-6c4a-4170-9dbc-0a4b1fdb2e77 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:49.393436 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b5ea1-6c4a-4170-9dbc-0a4b1fdb2e77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:49.394634 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b5ea1-6c4a-4170-9dbc-0a4b1fdb2e77 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=4717a71ca9d541939b2f0ba31daf6a0e, user.id=2d4b871d08f7414daa5606b8b7d3e0d1) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:49.395117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b5ea1-6c4a-4170-9dbc-0a4b1fdb2e77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:49.396955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b5ea1-6c4a-4170-9dbc-0a4b1fdb2e77 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:49.403812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 528/1059] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:42:49 2022] GET /identity/v3/role_assignments?user.id=2d4b871d08f7414daa5606b8b7d3e0d1&scope.project.id=4717a71ca9d541939b2f0ba31daf6a0e&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:49.489502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d18438-9e5c-4767-aa14-a9a4fd9b09c3 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:49.490249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d18438-9e5c-4767-aa14-a9a4fd9b09c3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:49.490603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d18438-9e5c-4767-aa14-a9a4fd9b09c3 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:49.493431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 532/1060] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:49.500132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c2089-2873-4231-8971-7ec46232a139 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:49.500509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c2089-2873-4231-8971-7ec46232a139 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:49.500765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c2089-2873-4231-8971-7ec46232a139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:49.527136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f56c2089-2873-4231-8971-7ec46232a139 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:49.528444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f56c2089-2873-4231-8971-7ec46232a139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:49.631149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 529/1061] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:49.653947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d18438-9e5c-4767-aa14-a9a4fd9b09c3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:49.807458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d18438-9e5c-4767-aa14-a9a4fd9b09c3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:49.854733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9b437-9b95-420a-ba2e-75c8c882e04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:49.855565 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9b437-9b95-420a-ba2e-75c8c882e04d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:49.855848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9b437-9b95-420a-ba2e-75c8c882e04d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:49.856121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9b437-9b95-420a-ba2e-75c8c882e04d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:49.867513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9b437-9b95-420a-ba2e-75c8c882e04d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-f8e9b437-9b95-420a-ba2e-75c8c882e04d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:49.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:49.877458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:49.878335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:49.886830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 533/1062] 10.222.0.79 () {56 vars in 1120 bytes} [Fri May 13 13:42:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:49.887078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c2089-2873-4231-8971-7ec46232a139 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:49.997770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c2089-2873-4231-8971-7ec46232a139 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:50.034210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff4354-1ae6-4cef-b74c-c685f88ea6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:50.034921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff4354-1ae6-4cef-b74c-c685f88ea6e9 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:50.035192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff4354-1ae6-4cef-b74c-c685f88ea6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:50.035472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff4354-1ae6-4cef-b74c-c685f88ea6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:50.036227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff4354-1ae6-4cef-b74c-c685f88ea6e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:50.036528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff4354-1ae6-4cef-b74c-c685f88ea6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:50.037909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff4354-1ae6-4cef-b74c-c685f88ea6e9 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:50.046741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 530/1063] 10.222.0.79 () {56 vars in 1115 bytes} [Fri May 13 13:42:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:50.053198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9b437-9b95-420a-ba2e-75c8c882e04d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:50.192508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9b437-9b95-420a-ba2e-75c8c882e04d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:50.228614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f58ca6-f0ce-4ded-b04d-5f499c3e0cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:50.229412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f58ca6-f0ce-4ded-b04d-5f499c3e0cdf None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:50.229698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f58ca6-f0ce-4ded-b04d-5f499c3e0cdf None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:50.229956 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f58ca6-f0ce-4ded-b04d-5f499c3e0cdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:50.242687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f58ca6-f0ce-4ded-b04d-5f499c3e0cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:50.244182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f58ca6-f0ce-4ded-b04d-5f499c3e0cdf None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-24f58ca6-f0ce-4ded-b04d-5f499c3e0cdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:50.263807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:50.264867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:50.265841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:50.265841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:50.265841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:50.265841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:50.270747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 534/1064] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:50.271680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff4354-1ae6-4cef-b74c-c685f88ea6e9 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:50.409880 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff4354-1ae6-4cef-b74c-c685f88ea6e9 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:50.459532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8e1c1-1d70-4e9e-80a4-b462bceeb17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:50.460120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8e1c1-1d70-4e9e-80a4-b462bceeb17a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:50.461438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8e1c1-1d70-4e9e-80a4-b462bceeb17a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:50.461438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8e1c1-1d70-4e9e-80a4-b462bceeb17a None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:50.461438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8e1c1-1d70-4e9e-80a4-b462bceeb17a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:50.461691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8e1c1-1d70-4e9e-80a4-b462bceeb17a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:50.463080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8e1c1-1d70-4e9e-80a4-b462bceeb17a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:50.481524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 531/1065] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:50.488932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f58ca6-f0ce-4ded-b04d-5f499c3e0cdf None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:50.649968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f58ca6-f0ce-4ded-b04d-5f499c3e0cdf None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:50.690948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e722c6-c7d2-444d-ae89-b556345d0807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:50.691855 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e722c6-c7d2-444d-ae89-b556345d0807 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:50.692199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e722c6-c7d2-444d-ae89-b556345d0807 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:50.692503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e722c6-c7d2-444d-ae89-b556345d0807 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:50.701174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e722c6-c7d2-444d-ae89-b556345d0807 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:50.702621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e722c6-c7d2-444d-ae89-b556345d0807 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-a4e722c6-c7d2-444d-ae89-b556345d0807 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:50.717523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:50.718627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:50.720251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:50.720251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:50.720251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:50.720251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:50.720251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:50.720251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:50.720251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:50.720251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:50.720251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:42:50.720251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:50.723096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 535/1066] 10.222.0.79 () {56 vars in 1093 bytes} [Fri May 13 13:42:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:50.724499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8e1c1-1d70-4e9e-80a4-b462bceeb17a None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:50.731750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3f8cf-abc1-47cd-bc5c-d44e38b2ca32 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:50.732127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3f8cf-abc1-47cd-bc5c-d44e38b2ca32 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:50.732354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3f8cf-abc1-47cd-bc5c-d44e38b2ca32 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:50.733911 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 536/1067] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:50.740101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fb6d2-a1a3-4df9-9075-82b6158adb69 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:50.740385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fb6d2-a1a3-4df9-9075-82b6158adb69 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:50.740660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fb6d2-a1a3-4df9-9075-82b6158adb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:50.760048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-724fb6d2-a1a3-4df9-9075-82b6158adb69 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:50.761273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-724fb6d2-a1a3-4df9-9075-82b6158adb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:50.838156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8e1c1-1d70-4e9e-80a4-b462bceeb17a None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:50.847064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 537/1068] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:50.870257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fb6d2-a1a3-4df9-9075-82b6158adb69 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:50.883692 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f1a41-6006-4483-afa3-1242ba0a6caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:50.884562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f1a41-6006-4483-afa3-1242ba0a6caa None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:50.885036 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f1a41-6006-4483-afa3-1242ba0a6caa None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:50.885448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f1a41-6006-4483-afa3-1242ba0a6caa None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:50.886360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f1a41-6006-4483-afa3-1242ba0a6caa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:50.886732 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f1a41-6006-4483-afa3-1242ba0a6caa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:50.888541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f1a41-6006-4483-afa3-1242ba0a6caa None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:50.908387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 532/1069] 10.222.0.79 () {56 vars in 1133 bytes} [Fri May 13 13:42:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:50.915550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f1a41-6006-4483-afa3-1242ba0a6caa None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:51.013260 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fb6d2-a1a3-4df9-9075-82b6158adb69 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:51.056814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f1a41-6006-4483-afa3-1242ba0a6caa None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:51.064359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad27b1-66bb-4828-a27b-a6ad7760382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:51.065021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad27b1-66bb-4828-a27b-a6ad7760382f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:51.065304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad27b1-66bb-4828-a27b-a6ad7760382f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:51.065578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad27b1-66bb-4828-a27b-a6ad7760382f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:51.101502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751408-8ffc-4bec-8e25-fa537f8100cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:51.102496 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751408-8ffc-4bec-8e25-fa537f8100cb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:51.102898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751408-8ffc-4bec-8e25-fa537f8100cb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:51.103206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec751408-8ffc-4bec-8e25-fa537f8100cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:51.117254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec751408-8ffc-4bec-8e25-fa537f8100cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:51.119029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec751408-8ffc-4bec-8e25-fa537f8100cb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-ec751408-8ffc-4bec-8e25-fa537f8100cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:51.139438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:51.140722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:51.141849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:51.141849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:51.141849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:51.141849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:51.141849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 533/1070] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:51.146270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751408-8ffc-4bec-8e25-fa537f8100cb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:51.148998 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad27b1-66bb-4828-a27b-a6ad7760382f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:51.150923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad27b1-66bb-4828-a27b-a6ad7760382f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:51.274862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec751408-8ffc-4bec-8e25-fa537f8100cb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:51.277125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 538/1071] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:42:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:42:51.284592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad27b1-66bb-4828-a27b-a6ad7760382f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:51.334469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cd5e4-6a39-459d-a5fc-8dbde1656a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:51.335483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cd5e4-6a39-459d-a5fc-8dbde1656a49 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:51.335902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cd5e4-6a39-459d-a5fc-8dbde1656a49 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:51.336362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cd5e4-6a39-459d-a5fc-8dbde1656a49 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:51.337318 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cd5e4-6a39-459d-a5fc-8dbde1656a49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:51.337773 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cd5e4-6a39-459d-a5fc-8dbde1656a49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:51.339737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cd5e4-6a39-459d-a5fc-8dbde1656a49 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:51.353321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 534/1072] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:51.359107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cd5e4-6a39-459d-a5fc-8dbde1656a49 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:51.416384 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad27b1-66bb-4828-a27b-a6ad7760382f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:51.460419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f99b-e23c-4955-992f-3b38b8b49fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:51.461238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f99b-e23c-4955-992f-3b38b8b49fa2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:51.461606 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f99b-e23c-4955-992f-3b38b8b49fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:51.461874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f99b-e23c-4955-992f-3b38b8b49fa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:51.475116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01f99b-e23c-4955-992f-3b38b8b49fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:51.476605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01f99b-e23c-4955-992f-3b38b8b49fa2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:51.486187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cd5e4-6a39-459d-a5fc-8dbde1656a49 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-3d01f99b-e23c-4955-992f-3b38b8b49fa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:51.496704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:51.497706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:51.498594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:51.498594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:51.498594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:51.498594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:51.503786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 539/1073] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:51.504806 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f99b-e23c-4955-992f-3b38b8b49fa2 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:51.528044 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80980-3328-4e9b-ae42-973f4dc21b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:51.528780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80980-3328-4e9b-ae42-973f4dc21b33 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:51.529075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80980-3328-4e9b-ae42-973f4dc21b33 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:51.529331 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80980-3328-4e9b-ae42-973f4dc21b33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:51.538016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb80980-3328-4e9b-ae42-973f4dc21b33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:51.539522 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb80980-3328-4e9b-ae42-973f4dc21b33 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-ecb80980-3328-4e9b-ae42-973f4dc21b33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:51.560887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:51.562108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:51.570550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:51.570550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:51.570550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:51.570550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:51.570550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:51.570550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:51.570550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:51.570550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:51.570550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 535/1074] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:51.573798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80980-3328-4e9b-ae42-973f4dc21b33 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:51.626021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f99b-e23c-4955-992f-3b38b8b49fa2 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:51.662358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187f29c-db35-494d-a5a4-03d3bf7f3277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:51.663167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187f29c-db35-494d-a5a4-03d3bf7f3277 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:51.663335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187f29c-db35-494d-a5a4-03d3bf7f3277 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:51.663641 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187f29c-db35-494d-a5a4-03d3bf7f3277 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:51.664332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187f29c-db35-494d-a5a4-03d3bf7f3277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:51.664636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187f29c-db35-494d-a5a4-03d3bf7f3277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:51.666013 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187f29c-db35-494d-a5a4-03d3bf7f3277 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:51.680425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 540/1075] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:51.690848 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80980-3328-4e9b-ae42-973f4dc21b33 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:51.738115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fb43d-ef4d-4210-ace0-14ccf682571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:51.740177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb43d-ef4d-4210-ace0-14ccf682571c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:51.741382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb43d-ef4d-4210-ace0-14ccf682571c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:51.741382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fb43d-ef4d-4210-ace0-14ccf682571c None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:51.742629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fb43d-ef4d-4210-ace0-14ccf682571c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:51.743205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fb43d-ef4d-4210-ace0-14ccf682571c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:51.745060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fb43d-ef4d-4210-ace0-14ccf682571c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:51.760135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 536/1076] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:51.766283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187f29c-db35-494d-a5a4-03d3bf7f3277 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:51.786855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e886-a882-4fa9-acc1-fd1d4d30289e None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:51.786855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e886-a882-4fa9-acc1-fd1d4d30289e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:51.786855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e886-a882-4fa9-acc1-fd1d4d30289e None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:51.786855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 537/1077] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:51.790951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fb5e2-273f-42ab-8056-8cfb5ba724ef None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:51.791435 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fb5e2-273f-42ab-8056-8cfb5ba724ef None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:51.791751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fb5e2-273f-42ab-8056-8cfb5ba724ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:51.818074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e18fb5e2-273f-42ab-8056-8cfb5ba724ef None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:51.819346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e18fb5e2-273f-42ab-8056-8cfb5ba724ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:51.894807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187f29c-db35-494d-a5a4-03d3bf7f3277 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:51.916864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 538/1078] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:51.937605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fb5e2-273f-42ab-8056-8cfb5ba724ef None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:51.943488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2652d1-e431-4f24-a129-99aaa0dc8b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:51.944406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2652d1-e431-4f24-a129-99aaa0dc8b6c None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:51.944685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2652d1-e431-4f24-a129-99aaa0dc8b6c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:51.944958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2652d1-e431-4f24-a129-99aaa0dc8b6c None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/24ad0dd9d265425087be2be8d245b7fc/roles/f4a90110e2c845408f5ec7685f7ef2d0` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:51.951116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2652d1-e431-4f24-a129-99aaa0dc8b6c None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=24ad0dd9d265425087be2be8d245b7fc, role_id=f4a90110e2c845408f5ec7685f7ef2d0)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:51.952577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2652d1-e431-4f24-a129-99aaa0dc8b6c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:51.969035 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 541/1079] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:42:51 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/24ad0dd9d265425087be2be8d245b7fc/roles/f4a90110e2c845408f5ec7685f7ef2d0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:42:52.073784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fb5e2-273f-42ab-8056-8cfb5ba724ef None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:52.112494 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51efe1-e086-41b0-abff-49d396768f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:52.113311 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51efe1-e086-41b0-abff-49d396768f95 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:52.113591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51efe1-e086-41b0-abff-49d396768f95 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:52.113866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51efe1-e086-41b0-abff-49d396768f95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:52.127513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51efe1-e086-41b0-abff-49d396768f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-4a51efe1-e086-41b0-abff-49d396768f95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:52.135707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:52.137185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:52.138352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 539/1080] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:52.144104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2652d1-e431-4f24-a129-99aaa0dc8b6c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:52.251727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2652d1-e431-4f24-a129-99aaa0dc8b6c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:52.286944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60379d-c2ea-4efa-bd1d-d2bdb8067fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:52.287629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60379d-c2ea-4efa-bd1d-d2bdb8067fc5 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:52.287910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60379d-c2ea-4efa-bd1d-d2bdb8067fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:52.288178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60379d-c2ea-4efa-bd1d-d2bdb8067fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:52.288916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60379d-c2ea-4efa-bd1d-d2bdb8067fc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:52.289216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60379d-c2ea-4efa-bd1d-d2bdb8067fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:52.290602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60379d-c2ea-4efa-bd1d-d2bdb8067fc5 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:52.300064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 542/1081] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:42:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:52.305927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51efe1-e086-41b0-abff-49d396768f95 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:52.418771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51efe1-e086-41b0-abff-49d396768f95 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:52.463080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300354e5-4d1f-4e32-9f67-ab19c7bfea14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:52.463976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300354e5-4d1f-4e32-9f67-ab19c7bfea14 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:52.464257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300354e5-4d1f-4e32-9f67-ab19c7bfea14 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:52.464530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300354e5-4d1f-4e32-9f67-ab19c7bfea14 None admin] PATH_INFO: `/v3/users/2d4b871d08f7414daa5606b8b7d3e0d1` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:52.465824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300354e5-4d1f-4e32-9f67-ab19c7bfea14 None admin] RBAC: Authorizing `identity:get_user(user_id=2d4b871d08f7414daa5606b8b7d3e0d1)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:52.467347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300354e5-4d1f-4e32-9f67-ab19c7bfea14 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:52.469029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 540/1082] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:42:52 2022] GET /identity/v3/users/2d4b871d08f7414daa5606b8b7d3e0d1 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:52.474754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60379d-c2ea-4efa-bd1d-d2bdb8067fc5 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:52.575737 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60379d-c2ea-4efa-bd1d-d2bdb8067fc5 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:52.615717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6a037-39f4-4184-8781-750a816c4899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:52.616597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6a037-39f4-4184-8781-750a816c4899 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:52.616872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6a037-39f4-4184-8781-750a816c4899 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:52.617141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6a037-39f4-4184-8781-750a816c4899 None admin] PATH_INFO: `/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:52.618412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6a037-39f4-4184-8781-750a816c4899 None admin] RBAC: Authorizing `identity:get_project(project_id=4717a71ca9d541939b2f0ba31daf6a0e)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:52.619898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6a037-39f4-4184-8781-750a816c4899 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:52.621627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 543/1083] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:42:52 2022] GET /identity/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:52.630084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300354e5-4d1f-4e32-9f67-ab19c7bfea14 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:52.731970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300354e5-4d1f-4e32-9f67-ab19c7bfea14 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:52.767557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034fdc6-1a7b-4fe9-bd41-49bab730452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:52.768443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034fdc6-1a7b-4fe9-bd41-49bab730452c None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:52.768718 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034fdc6-1a7b-4fe9-bd41-49bab730452c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:52.769053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034fdc6-1a7b-4fe9-bd41-49bab730452c None admin] PATH_INFO: `/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e/users/2d4b871d08f7414daa5606b8b7d3e0d1/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:52.771057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034fdc6-1a7b-4fe9-bd41-49bab730452c None admin] RBAC: Authorizing `identity:create_grant(project_id=4717a71ca9d541939b2f0ba31daf6a0e, user_id=2d4b871d08f7414daa5606b8b7d3e0d1, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:52.772495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034fdc6-1a7b-4fe9-bd41-49bab730452c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:52.787668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 541/1084] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:42:52 2022] PUT /identity/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e/users/2d4b871d08f7414daa5606b8b7d3e0d1/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:42:54.314134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444ad85-a807-4a5f-90a6-6df754c706d6 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:54.315127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444ad85-a807-4a5f-90a6-6df754c706d6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:54.315127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444ad85-a807-4a5f-90a6-6df754c706d6 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:54.316904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 544/1085] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:54.325180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc8fcf-3482-44eb-a993-328f9de74228 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:54.325757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc8fcf-3482-44eb-a993-328f9de74228 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:54.325757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc8fcf-3482-44eb-a993-328f9de74228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:54.346041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-74cc8fcf-3482-44eb-a993-328f9de74228 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:54.347862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-74cc8fcf-3482-44eb-a993-328f9de74228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:54.438530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 542/1086] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:54.469878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444ad85-a807-4a5f-90a6-6df754c706d6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:54.601795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444ad85-a807-4a5f-90a6-6df754c706d6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:54.646442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f657cf-76f6-4bb7-81fe-e92c06c7b874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:54.647295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f657cf-76f6-4bb7-81fe-e92c06c7b874 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:54.647582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f657cf-76f6-4bb7-81fe-e92c06c7b874 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:54.647856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f657cf-76f6-4bb7-81fe-e92c06c7b874 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:54.660749 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f657cf-76f6-4bb7-81fe-e92c06c7b874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-80f657cf-76f6-4bb7-81fe-e92c06c7b874 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:54.669119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:54.670515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:54.671410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 545/1087] 10.222.0.79 () {56 vars in 1120 bytes} [Fri May 13 13:42:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:54.676741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc8fcf-3482-44eb-a993-328f9de74228 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:54.795757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc8fcf-3482-44eb-a993-328f9de74228 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:54.836415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad657823-2d3e-41ed-b5a7-dd7d43c2d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:54.837092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad657823-2d3e-41ed-b5a7-dd7d43c2d99c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:54.838598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad657823-2d3e-41ed-b5a7-dd7d43c2d99c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:54.838598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad657823-2d3e-41ed-b5a7-dd7d43c2d99c None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:54.838598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad657823-2d3e-41ed-b5a7-dd7d43c2d99c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:54.839354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad657823-2d3e-41ed-b5a7-dd7d43c2d99c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:54.840161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad657823-2d3e-41ed-b5a7-dd7d43c2d99c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:54.849665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 543/1088] 10.222.0.79 () {56 vars in 1115 bytes} [Fri May 13 13:42:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:54.856052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f657cf-76f6-4bb7-81fe-e92c06c7b874 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:54.977081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f657cf-76f6-4bb7-81fe-e92c06c7b874 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:55.021790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5de16-f275-4c54-a6d7-a38a4996c5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:55.022794 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5de16-f275-4c54-a6d7-a38a4996c5c8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:55.023077 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5de16-f275-4c54-a6d7-a38a4996c5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:55.023342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5de16-f275-4c54-a6d7-a38a4996c5c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:55.034803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5de16-f275-4c54-a6d7-a38a4996c5c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:55.036397 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5de16-f275-4c54-a6d7-a38a4996c5c8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-5ae5de16-f275-4c54-a6d7-a38a4996c5c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:55.055830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:55.056967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:55.058090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:55.058090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:55.058090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:55.058090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:55.058090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 546/1089] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:55.063855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad657823-2d3e-41ed-b5a7-dd7d43c2d99c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:55.187761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad657823-2d3e-41ed-b5a7-dd7d43c2d99c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:55.190493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa327c-c602-4902-8a32-821e4c68d1c6 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:55.190806 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa327c-c602-4902-8a32-821e4c68d1c6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:55.191087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa327c-c602-4902-8a32-821e4c68d1c6 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:55.192463 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 547/1090] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:42:55.199136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9a6c6-e0d5-45a0-ac6c-9ffe9a2715eb None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:55.199337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9a6c6-e0d5-45a0-ac6c-9ffe9a2715eb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:55.199514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9a6c6-e0d5-45a0-ac6c-9ffe9a2715eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:55.216563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a9a9a6c6-e0d5-45a0-ac6c-9ffe9a2715eb None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:55.217801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a9a9a6c6-e0d5-45a0-ac6c-9ffe9a2715eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:55.225681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48246f1-cf22-4be3-8e12-c56f291b51b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:55.226356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48246f1-cf22-4be3-8e12-c56f291b51b5 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:55.226636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48246f1-cf22-4be3-8e12-c56f291b51b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:55.226962 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48246f1-cf22-4be3-8e12-c56f291b51b5 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:55.227693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48246f1-cf22-4be3-8e12-c56f291b51b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:55.228000 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48246f1-cf22-4be3-8e12-c56f291b51b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:55.229424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48246f1-cf22-4be3-8e12-c56f291b51b5 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:55.242349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 544/1091] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:55.248266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48246f1-cf22-4be3-8e12-c56f291b51b5 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:55.295060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 548/1092] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:42:55.322858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9a6c6-e0d5-45a0-ac6c-9ffe9a2715eb None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:55.377926 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48246f1-cf22-4be3-8e12-c56f291b51b5 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:55.432361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae763e18-ed34-4a94-b3d6-06b5eeea6e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:55.433822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae763e18-ed34-4a94-b3d6-06b5eeea6e13 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:55.434320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae763e18-ed34-4a94-b3d6-06b5eeea6e13 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:55.434863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae763e18-ed34-4a94-b3d6-06b5eeea6e13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:55.443401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9a6c6-e0d5-45a0-ac6c-9ffe9a2715eb None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:55.450050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae763e18-ed34-4a94-b3d6-06b5eeea6e13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:55.453000 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae763e18-ed34-4a94-b3d6-06b5eeea6e13 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-ae763e18-ed34-4a94-b3d6-06b5eeea6e13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:42:55.480578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:55.481640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:55.482598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:55.482598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:55.482598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:42:55.482598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:42:55.482598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:42:55.482598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:42:55.482598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:42:55.482598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:42:55.482598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:42:55.482598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:55.483637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 545/1093] 10.222.0.79 () {56 vars in 1093 bytes} [Fri May 13 13:42:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:55.492405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae763e18-ed34-4a94-b3d6-06b5eeea6e13 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:55.495760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812f723-d292-4a42-a167-6f63f83702c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:55.497132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812f723-d292-4a42-a167-6f63f83702c3 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:55.497599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812f723-d292-4a42-a167-6f63f83702c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:55.498084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812f723-d292-4a42-a167-6f63f83702c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:55.519731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812f723-d292-4a42-a167-6f63f83702c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-d812f723-d292-4a42-a167-6f63f83702c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:42:55.528050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:55.529966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:42:55.531644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 549/1094] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:42:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:42:55.538922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812f723-d292-4a42-a167-6f63f83702c3 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:55.625674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae763e18-ed34-4a94-b3d6-06b5eeea6e13 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:55.665940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e50c9-2d86-4787-9185-19985439dca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:55.666670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e50c9-2d86-4787-9185-19985439dca7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:55.666986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e50c9-2d86-4787-9185-19985439dca7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:55.667264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e50c9-2d86-4787-9185-19985439dca7 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:55.668121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e50c9-2d86-4787-9185-19985439dca7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:55.668432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e50c9-2d86-4787-9185-19985439dca7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:55.669921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e50c9-2d86-4787-9185-19985439dca7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:55.676752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812f723-d292-4a42-a167-6f63f83702c3 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:55.689456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 546/1095] 10.222.0.79 () {56 vars in 1133 bytes} [Fri May 13 13:42:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:55.695707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e50c9-2d86-4787-9185-19985439dca7 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:55.726031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e0833-3ccd-4fd1-a495-c93bb7a2e627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:55.726881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e0833-3ccd-4fd1-a495-c93bb7a2e627 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:55.727169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e0833-3ccd-4fd1-a495-c93bb7a2e627 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:55.727444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e0833-3ccd-4fd1-a495-c93bb7a2e627 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:55.728410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e0833-3ccd-4fd1-a495-c93bb7a2e627 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:55.728724 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e0833-3ccd-4fd1-a495-c93bb7a2e627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:55.730136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e0833-3ccd-4fd1-a495-c93bb7a2e627 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:55.745053 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 550/1096] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:42:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:55.751905 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e0833-3ccd-4fd1-a495-c93bb7a2e627 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:55.829101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e50c9-2d86-4787-9185-19985439dca7 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:55.882483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e0833-3ccd-4fd1-a495-c93bb7a2e627 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:55.889290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ca7ab-f6a1-490c-80e2-b6e63bab2cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:55.889947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ca7ab-f6a1-490c-80e2-b6e63bab2cec None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:55.890224 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ca7ab-f6a1-490c-80e2-b6e63bab2cec None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:55.890500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ca7ab-f6a1-490c-80e2-b6e63bab2cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:55.925004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3bb9e-59f6-4d16-a3c1-b60459734221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:55.926457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3bb9e-59f6-4d16-a3c1-b60459734221 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:55.926644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3bb9e-59f6-4d16-a3c1-b60459734221 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:55.926972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3bb9e-59f6-4d16-a3c1-b60459734221 None admin] PATH_INFO: `/v3/users/2d4b871d08f7414daa5606b8b7d3e0d1` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:55.928757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3bb9e-59f6-4d16-a3c1-b60459734221 None admin] RBAC: Authorizing `identity:get_user(user_id=2d4b871d08f7414daa5606b8b7d3e0d1)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:55.930624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3bb9e-59f6-4d16-a3c1-b60459734221 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:55.932405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 551/1097] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:42:55 2022] GET /identity/v3/users/2d4b871d08f7414daa5606b8b7d3e0d1 => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:55.939177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3bb9e-59f6-4d16-a3c1-b60459734221 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:55.981701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ca7ab-f6a1-490c-80e2-b6e63bab2cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:55.983237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ca7ab-f6a1-490c-80e2-b6e63bab2cec None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:56.050910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3bb9e-59f6-4d16-a3c1-b60459734221 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:56.091274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 547/1098] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:42:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:42:56.092792 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff888a7e-5c39-4772-9290-1b21cd757214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:56.093899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff888a7e-5c39-4772-9290-1b21cd757214 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:56.094313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff888a7e-5c39-4772-9290-1b21cd757214 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:56.094750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff888a7e-5c39-4772-9290-1b21cd757214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:56.097453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ca7ab-f6a1-490c-80e2-b6e63bab2cec None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:56.188190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff888a7e-5c39-4772-9290-1b21cd757214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:56.189674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff888a7e-5c39-4772-9290-1b21cd757214 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:56.220898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ca7ab-f6a1-490c-80e2-b6e63bab2cec None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:56.271036 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc46cf-6c2f-445d-87db-023e28815ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:56.271844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc46cf-6c2f-445d-87db-023e28815ecf None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:56.272121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc46cf-6c2f-445d-87db-023e28815ecf None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:56.272397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc46cf-6c2f-445d-87db-023e28815ecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:56.284231 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc46cf-6c2f-445d-87db-023e28815ecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:56.285716 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc46cf-6c2f-445d-87db-023e28815ecf None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-f0fc46cf-6c2f-445d-87db-023e28815ecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:42:56.304423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:56.305424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:56.306459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:42:56.306459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:42:56.306459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:42:56.306459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:56.307143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 548/1099] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:56.312183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc46cf-6c2f-445d-87db-023e28815ecf None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:56.321220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 552/1100] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:42:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:42:56.327456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff888a7e-5c39-4772-9290-1b21cd757214 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:56.424272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc46cf-6c2f-445d-87db-023e28815ecf None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:56.444414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff888a7e-5c39-4772-9290-1b21cd757214 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:56.462980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622ed73-72f2-4b60-bed5-436506e7663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:56.463642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622ed73-72f2-4b60-bed5-436506e7663f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:56.463916 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622ed73-72f2-4b60-bed5-436506e7663f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:56.464188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622ed73-72f2-4b60-bed5-436506e7663f None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:56.464905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622ed73-72f2-4b60-bed5-436506e7663f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:56.465211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622ed73-72f2-4b60-bed5-436506e7663f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:56.466596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622ed73-72f2-4b60-bed5-436506e7663f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:56.480837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 549/1101] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:56.490490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622ed73-72f2-4b60-bed5-436506e7663f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:56.492445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fe872-5a02-4f6c-a520-67707e503125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:56.493961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fe872-5a02-4f6c-a520-67707e503125 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:56.494773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fe872-5a02-4f6c-a520-67707e503125 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:56.495287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fe872-5a02-4f6c-a520-67707e503125 None admin] PATH_INFO: `/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:56.498301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fe872-5a02-4f6c-a520-67707e503125 None admin] RBAC: Authorizing `identity:get_project(project_id=4717a71ca9d541939b2f0ba31daf6a0e)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:56.503418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fe872-5a02-4f6c-a520-67707e503125 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:56.505953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 553/1102] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:42:56 2022] GET /identity/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:56.512073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fe872-5a02-4f6c-a520-67707e503125 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:56.628762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622ed73-72f2-4b60-bed5-436506e7663f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:56.632277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fe872-5a02-4f6c-a520-67707e503125 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:56.674611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22256fba-1d8e-4209-a26d-e73c05a03bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:56.675454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22256fba-1d8e-4209-a26d-e73c05a03bc0 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:56.675736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22256fba-1d8e-4209-a26d-e73c05a03bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:56.676006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22256fba-1d8e-4209-a26d-e73c05a03bc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:56.676446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2d8c3-5e2e-4757-929d-ba4007a86420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:56.677134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2d8c3-5e2e-4757-929d-ba4007a86420 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:56.677419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2d8c3-5e2e-4757-929d-ba4007a86420 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:56.677843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2d8c3-5e2e-4757-929d-ba4007a86420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:56.678512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2d8c3-5e2e-4757-929d-ba4007a86420 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=4717a71ca9d541939b2f0ba31daf6a0e, user.id=2d4b871d08f7414daa5606b8b7d3e0d1) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:56.678870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2d8c3-5e2e-4757-929d-ba4007a86420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:56.680373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2d8c3-5e2e-4757-929d-ba4007a86420 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:56.684194 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22256fba-1d8e-4209-a26d-e73c05a03bc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:56.685661 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22256fba-1d8e-4209-a26d-e73c05a03bc0 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:56.686872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 554/1103] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:42:56 2022] GET /identity/v3/role_assignments?user.id=2d4b871d08f7414daa5606b8b7d3e0d1&scope.project.id=4717a71ca9d541939b2f0ba31daf6a0e&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-22256fba-1d8e-4209-a26d-e73c05a03bc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:42:56.701639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:42:56.702865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:42:56.704828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:42:56.704828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:42:56.704828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:42:56.704828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:42:56.704828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:42:56.704828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:42:56.704828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:42:56.704828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:42:56.704828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 550/1104] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:42:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:42:56.709862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2d8c3-5e2e-4757-929d-ba4007a86420 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:56.827869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2d8c3-5e2e-4757-929d-ba4007a86420 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:56.873865 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caeb6c7-bdb0-46e6-a488-cd3cd6ac66e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:56.874543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caeb6c7-bdb0-46e6-a488-cd3cd6ac66e0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:56.874852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caeb6c7-bdb0-46e6-a488-cd3cd6ac66e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:56.875120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caeb6c7-bdb0-46e6-a488-cd3cd6ac66e0 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:56.876502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caeb6c7-bdb0-46e6-a488-cd3cd6ac66e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:56.876799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caeb6c7-bdb0-46e6-a488-cd3cd6ac66e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:56.878390 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caeb6c7-bdb0-46e6-a488-cd3cd6ac66e0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:56.892852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 555/1105] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:42:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:56.900777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22256fba-1d8e-4209-a26d-e73c05a03bc0 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:57.028413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22256fba-1d8e-4209-a26d-e73c05a03bc0 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:57.069162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c16595-5fcf-45f7-8b73-38c17f182117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:57.069815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c16595-5fcf-45f7-8b73-38c17f182117 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:57.070092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c16595-5fcf-45f7-8b73-38c17f182117 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:57.070360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c16595-5fcf-45f7-8b73-38c17f182117 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:57.071233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c16595-5fcf-45f7-8b73-38c17f182117 None admin] RBAC: Adding query filter params (role.id=f4a90110e2c845408f5ec7685f7ef2d0, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=24ad0dd9d265425087be2be8d245b7fc) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:42:57.071604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c16595-5fcf-45f7-8b73-38c17f182117 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:57.073326 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c16595-5fcf-45f7-8b73-38c17f182117 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:57.080017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 551/1106] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:42:56 2022] GET /identity/v3/role_assignments?user.id=24ad0dd9d265425087be2be8d245b7fc&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=f4a90110e2c845408f5ec7685f7ef2d0 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:59.102341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221131e8-5af1-4c4f-83bb-21f14e17785a None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:59.102930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221131e8-5af1-4c4f-83bb-21f14e17785a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:59.102930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221131e8-5af1-4c4f-83bb-21f14e17785a None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:59.104270 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 556/1107] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:42:59.109971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696178cb-e2ad-4dc3-89d3-888b5309f401 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:59.110502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696178cb-e2ad-4dc3-89d3-888b5309f401 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:59.110502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696178cb-e2ad-4dc3-89d3-888b5309f401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:59.128632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-696178cb-e2ad-4dc3-89d3-888b5309f401 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:42:59.130096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-696178cb-e2ad-4dc3-89d3-888b5309f401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:42:59.213723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 552/1108] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:42:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:42:59.239484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221131e8-5af1-4c4f-83bb-21f14e17785a None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:59.371335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221131e8-5af1-4c4f-83bb-21f14e17785a None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:59.410988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db5e9d-5097-4d9d-a91c-92cc08bbdeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:59.411778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db5e9d-5097-4d9d-a91c-92cc08bbdeed None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:59.412058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db5e9d-5097-4d9d-a91c-92cc08bbdeed None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:59.412329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db5e9d-5097-4d9d-a91c-92cc08bbdeed None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:59.413518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db5e9d-5097-4d9d-a91c-92cc08bbdeed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:59.415004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db5e9d-5097-4d9d-a91c-92cc08bbdeed None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:59.416676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 557/1109] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:42:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:42:59.422564 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696178cb-e2ad-4dc3-89d3-888b5309f401 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:42:59.530096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696178cb-e2ad-4dc3-89d3-888b5309f401 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:42:59.567992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44238534-c876-40c6-8cca-d08725b07b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:42:59.568783 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44238534-c876-40c6-8cca-d08725b07b5f None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:42:59.568941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44238534-c876-40c6-8cca-d08725b07b5f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:42:59.569211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44238534-c876-40c6-8cca-d08725b07b5f None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:42:59.570016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44238534-c876-40c6-8cca-d08725b07b5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:42:59.571405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44238534-c876-40c6-8cca-d08725b07b5f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:42:59.595661 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 553/1110] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:42:59 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:43:02.008042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753fe6e-f6b2-4477-b4a4-523e02afe0a5 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:02.008042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753fe6e-f6b2-4477-b4a4-523e02afe0a5 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:02.008042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753fe6e-f6b2-4477-b4a4-523e02afe0a5 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:02.010562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 558/1111] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:02.016215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a745e0-9f79-4b44-9e9a-7ee644bb2eee None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:02.016835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a745e0-9f79-4b44-9e9a-7ee644bb2eee None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:02.016835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a745e0-9f79-4b44-9e9a-7ee644bb2eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:02.037332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-12a745e0-9f79-4b44-9e9a-7ee644bb2eee None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:02.038552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-12a745e0-9f79-4b44-9e9a-7ee644bb2eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:02.107961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 554/1112] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:43:02.133071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753fe6e-f6b2-4477-b4a4-523e02afe0a5 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:02.251812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753fe6e-f6b2-4477-b4a4-523e02afe0a5 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:02.285919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f179939-764f-4335-be3c-1f2ab5e25808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:02.286804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f179939-764f-4335-be3c-1f2ab5e25808 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:02.287082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f179939-764f-4335-be3c-1f2ab5e25808 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:02.287352 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f179939-764f-4335-be3c-1f2ab5e25808 None admin] PATH_INFO: `/v3/roles/4501501962414ed0abfba693ba95918c` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:02.288852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f179939-764f-4335-be3c-1f2ab5e25808 None admin] RBAC: Authorizing `identity:get_role(role_id=4501501962414ed0abfba693ba95918c)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:02.290221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f179939-764f-4335-be3c-1f2ab5e25808 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:02.291634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 559/1113] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:02 2022] GET /identity/v3/roles/4501501962414ed0abfba693ba95918c => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:02.297780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a745e0-9f79-4b44-9e9a-7ee644bb2eee None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:02.412208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a745e0-9f79-4b44-9e9a-7ee644bb2eee None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:02.452730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b41f6-5613-47a2-bec3-91783b938347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:02.453671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b41f6-5613-47a2-bec3-91783b938347 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:02.453947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b41f6-5613-47a2-bec3-91783b938347 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:02.454241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b41f6-5613-47a2-bec3-91783b938347 None admin] PATH_INFO: `/v3/users/09b44c08314349a5a9beb4bada466dac` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:02.468803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b41f6-5613-47a2-bec3-91783b938347 None admin] RBAC: Authorizing `identity:get_user(user_id=09b44c08314349a5a9beb4bada466dac)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:02.470854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b41f6-5613-47a2-bec3-91783b938347 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:02.472664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 555/1114] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:02 2022] GET /identity/v3/users/09b44c08314349a5a9beb4bada466dac => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:02.478746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f179939-764f-4335-be3c-1f2ab5e25808 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:02.586897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f179939-764f-4335-be3c-1f2ab5e25808 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:02.623299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13282cc-b30e-452c-9d06-fc294f49494f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:02.623951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13282cc-b30e-452c-9d06-fc294f49494f None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:02.624232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13282cc-b30e-452c-9d06-fc294f49494f None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:02.624506 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13282cc-b30e-452c-9d06-fc294f49494f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:02.699727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13282cc-b30e-452c-9d06-fc294f49494f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:02.701195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13282cc-b30e-452c-9d06-fc294f49494f None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:02.817642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 560/1115] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:43:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:43:02.824252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b41f6-5613-47a2-bec3-91783b938347 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:02.950180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b41f6-5613-47a2-bec3-91783b938347 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:02.990031 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab160852-aa43-4d3c-9123-63f71eb274bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:02.990944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab160852-aa43-4d3c-9123-63f71eb274bd None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:02.991226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab160852-aa43-4d3c-9123-63f71eb274bd None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:02.991501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab160852-aa43-4d3c-9123-63f71eb274bd None admin] PATH_INFO: `/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:02.992805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab160852-aa43-4d3c-9123-63f71eb274bd None admin] RBAC: Authorizing `identity:get_project(project_id=4717a71ca9d541939b2f0ba31daf6a0e)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:02.994308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab160852-aa43-4d3c-9123-63f71eb274bd None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:02.996186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 556/1116] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:43:02 2022] GET /identity/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:03.002698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13282cc-b30e-452c-9d06-fc294f49494f None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:03.113329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13282cc-b30e-452c-9d06-fc294f49494f None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:03.148676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d02b3-fe0e-4db5-90b5-8e2e953bb93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:03.149336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d02b3-fe0e-4db5-90b5-8e2e953bb93c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:03.149620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d02b3-fe0e-4db5-90b5-8e2e953bb93c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:03.149892 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d02b3-fe0e-4db5-90b5-8e2e953bb93c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:03.150746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d02b3-fe0e-4db5-90b5-8e2e953bb93c None admin] RBAC: Adding query filter params (role.id=4501501962414ed0abfba693ba95918c, scope.project.id=4717a71ca9d541939b2f0ba31daf6a0e, user.id=09b44c08314349a5a9beb4bada466dac) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:03.151061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d02b3-fe0e-4db5-90b5-8e2e953bb93c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:03.152501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d02b3-fe0e-4db5-90b5-8e2e953bb93c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:03.158428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 561/1117] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:43:03 2022] GET /identity/v3/role_assignments?user.id=09b44c08314349a5a9beb4bada466dac&scope.project.id=4717a71ca9d541939b2f0ba31daf6a0e&role.id=4501501962414ed0abfba693ba95918c => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:05.628966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecd8be-30e0-4fdc-b1e6-3327dfdaa2f9 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:05.629718 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecd8be-30e0-4fdc-b1e6-3327dfdaa2f9 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:05.629718 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecd8be-30e0-4fdc-b1e6-3327dfdaa2f9 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:05.630952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 557/1118] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:05.639069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea7854-ba8d-4177-b236-3c842c1c1b74 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:05.639874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea7854-ba8d-4177-b236-3c842c1c1b74 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:05.639874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea7854-ba8d-4177-b236-3c842c1c1b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:05.664207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-88ea7854-ba8d-4177-b236-3c842c1c1b74 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:05.665832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-88ea7854-ba8d-4177-b236-3c842c1c1b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:05.742462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 562/1119] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:43:05.766491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecd8be-30e0-4fdc-b1e6-3327dfdaa2f9 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:05.905570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecd8be-30e0-4fdc-b1e6-3327dfdaa2f9 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:05.953825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7e194-66de-4e35-8378-be089b1d0872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:05.955072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e194-66de-4e35-8378-be089b1d0872 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:05.955459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e194-66de-4e35-8378-be089b1d0872 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:05.955867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e194-66de-4e35-8378-be089b1d0872 None admin] PATH_INFO: `/v3/roles/4501501962414ed0abfba693ba95918c` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:05.957951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7e194-66de-4e35-8378-be089b1d0872 None admin] RBAC: Authorizing `identity:get_role(role_id=4501501962414ed0abfba693ba95918c)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:05.959987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7e194-66de-4e35-8378-be089b1d0872 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:05.961987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 558/1120] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:05 2022] GET /identity/v3/roles/4501501962414ed0abfba693ba95918c => generated 223 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:05.967932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea7854-ba8d-4177-b236-3c842c1c1b74 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:06.098213 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea7854-ba8d-4177-b236-3c842c1c1b74 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:06.145361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999dc95-41a6-4f97-8fb4-962e73918cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:06.146608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999dc95-41a6-4f97-8fb4-962e73918cb7 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:06.147036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999dc95-41a6-4f97-8fb4-962e73918cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:06.147423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999dc95-41a6-4f97-8fb4-962e73918cb7 None admin] PATH_INFO: `/v3/users/09b44c08314349a5a9beb4bada466dac` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:06.149108 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999dc95-41a6-4f97-8fb4-962e73918cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=09b44c08314349a5a9beb4bada466dac)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:06.151237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999dc95-41a6-4f97-8fb4-962e73918cb7 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:06.153588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 563/1121] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:05 2022] GET /identity/v3/users/09b44c08314349a5a9beb4bada466dac => generated 286 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:06.159762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7e194-66de-4e35-8378-be089b1d0872 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:06.272580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7e194-66de-4e35-8378-be089b1d0872 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:06.308788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75925f-93bd-45f4-a8b0-5ceeb7bef9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:06.309654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75925f-93bd-45f4-a8b0-5ceeb7bef9c7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:06.309937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75925f-93bd-45f4-a8b0-5ceeb7bef9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:06.310215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75925f-93bd-45f4-a8b0-5ceeb7bef9c7 None admin] PATH_INFO: `/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:06.311542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75925f-93bd-45f4-a8b0-5ceeb7bef9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=4717a71ca9d541939b2f0ba31daf6a0e)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:06.313623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75925f-93bd-45f4-a8b0-5ceeb7bef9c7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:06.315441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 559/1122] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:43:06 2022] GET /identity/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:43:06.321603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999dc95-41a6-4f97-8fb4-962e73918cb7 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:06.449456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999dc95-41a6-4f97-8fb4-962e73918cb7 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:06.490640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b51076-cab6-46d9-af36-6ef5fd7a1be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:06.491969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b51076-cab6-46d9-af36-6ef5fd7a1be7 None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:06.492422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b51076-cab6-46d9-af36-6ef5fd7a1be7 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:06.492738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b51076-cab6-46d9-af36-6ef5fd7a1be7 None admin] PATH_INFO: `/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e/users/09b44c08314349a5a9beb4bada466dac/roles/4501501962414ed0abfba693ba95918c` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:06.495987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b51076-cab6-46d9-af36-6ef5fd7a1be7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4717a71ca9d541939b2f0ba31daf6a0e, user_id=09b44c08314349a5a9beb4bada466dac, role_id=4501501962414ed0abfba693ba95918c)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:06.498082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b51076-cab6-46d9-af36-6ef5fd7a1be7 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:06.514101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 564/1123] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:43:06 2022] PUT /identity/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e/users/09b44c08314349a5a9beb4bada466dac/roles/4501501962414ed0abfba693ba95918c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:43:08.803973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe256c8-85b4-4cd2-a6c1-6efd387fd36a None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:08.804574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe256c8-85b4-4cd2-a6c1-6efd387fd36a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:08.804574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe256c8-85b4-4cd2-a6c1-6efd387fd36a None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:08.805882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 560/1124] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:08.811645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ccf8f-021e-4876-bf34-d3178452a9ef None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:08.812206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ccf8f-021e-4876-bf34-d3178452a9ef None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:08.812308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ccf8f-021e-4876-bf34-d3178452a9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:08.838006 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e08ccf8f-021e-4876-bf34-d3178452a9ef None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:08.839823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e08ccf8f-021e-4876-bf34-d3178452a9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:08.928001 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 565/1125] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:43:08.956994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe256c8-85b4-4cd2-a6c1-6efd387fd36a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:09.069237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe256c8-85b4-4cd2-a6c1-6efd387fd36a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:09.106578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675966c-e2c2-4428-b5c8-468572063cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:09.107844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675966c-e2c2-4428-b5c8-468572063cf3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:09.108258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675966c-e2c2-4428-b5c8-468572063cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:09.108672 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675966c-e2c2-4428-b5c8-468572063cf3 None admin] PATH_INFO: `/v3/roles/4501501962414ed0abfba693ba95918c` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:09.111365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675966c-e2c2-4428-b5c8-468572063cf3 None admin] RBAC: Authorizing `identity:get_role(role_id=4501501962414ed0abfba693ba95918c)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:09.113550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675966c-e2c2-4428-b5c8-468572063cf3 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:09.115642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 561/1126] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:08 2022] GET /identity/v3/roles/4501501962414ed0abfba693ba95918c => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:43:09.122151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ccf8f-021e-4876-bf34-d3178452a9ef None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:09.249651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ccf8f-021e-4876-bf34-d3178452a9ef None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:09.290564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5cf7c-6df3-43d2-bedb-eb94b2a844e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:09.291470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5cf7c-6df3-43d2-bedb-eb94b2a844e2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:09.291756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5cf7c-6df3-43d2-bedb-eb94b2a844e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:09.292042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5cf7c-6df3-43d2-bedb-eb94b2a844e2 None admin] PATH_INFO: `/v3/users/09b44c08314349a5a9beb4bada466dac` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:09.293305 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5cf7c-6df3-43d2-bedb-eb94b2a844e2 None admin] RBAC: Authorizing `identity:get_user(user_id=09b44c08314349a5a9beb4bada466dac)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:09.294775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5cf7c-6df3-43d2-bedb-eb94b2a844e2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:09.296450 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 566/1127] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:09 2022] GET /identity/v3/users/09b44c08314349a5a9beb4bada466dac => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:09.303092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675966c-e2c2-4428-b5c8-468572063cf3 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:09.424607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675966c-e2c2-4428-b5c8-468572063cf3 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:09.475202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed188e9-6fb5-4ab4-a75f-7a93d90792e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:09.475907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed188e9-6fb5-4ab4-a75f-7a93d90792e4 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:09.476188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed188e9-6fb5-4ab4-a75f-7a93d90792e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:09.476496 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed188e9-6fb5-4ab4-a75f-7a93d90792e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:09.553586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed188e9-6fb5-4ab4-a75f-7a93d90792e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:09.555086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed188e9-6fb5-4ab4-a75f-7a93d90792e4 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:09.683340 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 562/1128] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:43:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:43:09.689926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5cf7c-6df3-43d2-bedb-eb94b2a844e2 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:09.832489 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5cf7c-6df3-43d2-bedb-eb94b2a844e2 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:09.879042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafd3e1-5754-4054-8e2c-034745b6c589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:09.879909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafd3e1-5754-4054-8e2c-034745b6c589 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:09.880191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafd3e1-5754-4054-8e2c-034745b6c589 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:09.880467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafd3e1-5754-4054-8e2c-034745b6c589 None admin] PATH_INFO: `/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:09.881654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafd3e1-5754-4054-8e2c-034745b6c589 None admin] RBAC: Authorizing `identity:get_project(project_id=4717a71ca9d541939b2f0ba31daf6a0e)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:09.883118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafd3e1-5754-4054-8e2c-034745b6c589 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:09.884827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 567/1129] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:43:09 2022] GET /identity/v3/projects/4717a71ca9d541939b2f0ba31daf6a0e => generated 311 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:09.891693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed188e9-6fb5-4ab4-a75f-7a93d90792e4 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:10.016596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed188e9-6fb5-4ab4-a75f-7a93d90792e4 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:10.054790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefa0f5-0732-4c83-8e90-dc698797577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:10.055450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa0f5-0732-4c83-8e90-dc698797577d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:10.055727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa0f5-0732-4c83-8e90-dc698797577d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:10.055999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa0f5-0732-4c83-8e90-dc698797577d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:10.056827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefa0f5-0732-4c83-8e90-dc698797577d None admin] RBAC: Adding query filter params (role.id=4501501962414ed0abfba693ba95918c, scope.project.id=4717a71ca9d541939b2f0ba31daf6a0e, user.id=09b44c08314349a5a9beb4bada466dac) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:10.057133 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefa0f5-0732-4c83-8e90-dc698797577d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:10.058632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefa0f5-0732-4c83-8e90-dc698797577d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:10.065634 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 563/1130] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:43:09 2022] GET /identity/v3/role_assignments?user.id=09b44c08314349a5a9beb4bada466dac&scope.project.id=4717a71ca9d541939b2f0ba31daf6a0e&role.id=4501501962414ed0abfba693ba95918c => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:12.661592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13670a1-1c96-45b5-a4d2-d9aaee336019 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:12.662266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13670a1-1c96-45b5-a4d2-d9aaee336019 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:12.662266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13670a1-1c96-45b5-a4d2-d9aaee336019 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:12.663543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 568/1131] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:12.669836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20a4fc-6613-46f1-9958-e009276baeb8 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:12.670390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20a4fc-6613-46f1-9958-e009276baeb8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:12.670390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20a4fc-6613-46f1-9958-e009276baeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:12.688703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-bd20a4fc-6613-46f1-9958-e009276baeb8 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:12.689918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-bd20a4fc-6613-46f1-9958-e009276baeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:12.767126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 564/1132] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:43:12.791239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13670a1-1c96-45b5-a4d2-d9aaee336019 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:12.916391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13670a1-1c96-45b5-a4d2-d9aaee336019 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:12.953471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3c6e2-9a40-4e93-ac22-d25d64c2f8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:12.954279 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c6e2-9a40-4e93-ac22-d25d64c2f8e6 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:12.954556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c6e2-9a40-4e93-ac22-d25d64c2f8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:12.954869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c6e2-9a40-4e93-ac22-d25d64c2f8e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:12.956125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3c6e2-9a40-4e93-ac22-d25d64c2f8e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:12.957592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3c6e2-9a40-4e93-ac22-d25d64c2f8e6 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:12.959303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 569/1133] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:43:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:43:12.965234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20a4fc-6613-46f1-9958-e009276baeb8 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:13.106446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20a4fc-6613-46f1-9958-e009276baeb8 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:13.164493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328108b-3573-4535-ae47-99987f54a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:13.165205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328108b-3573-4535-ae47-99987f54a604 None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:13.165514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328108b-3573-4535-ae47-99987f54a604 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:13.165816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328108b-3573-4535-ae47-99987f54a604 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:13.166604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328108b-3573-4535-ae47-99987f54a604 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:13.168122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328108b-3573-4535-ae47-99987f54a604 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:13.204595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 565/1134] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:43:12 2022] POST /identity/v3/projects => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:43:15.574021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c84e84-dec4-4a28-adac-9c5f3c19c382 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:15.574609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c84e84-dec4-4a28-adac-9c5f3c19c382 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:15.574609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c84e84-dec4-4a28-adac-9c5f3c19c382 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:15.576115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 570/1135] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:15.585039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51b032-a7c7-4aa0-891e-4abdb99373b7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:15.585635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51b032-a7c7-4aa0-891e-4abdb99373b7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:15.585840 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51b032-a7c7-4aa0-891e-4abdb99373b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:15.607326 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-ac51b032-a7c7-4aa0-891e-4abdb99373b7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:15.608536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ac51b032-a7c7-4aa0-891e-4abdb99373b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:15.677941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 566/1136] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:43:15.703777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c84e84-dec4-4a28-adac-9c5f3c19c382 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:15.812186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c84e84-dec4-4a28-adac-9c5f3c19c382 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:15.845830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841bd23-f5d1-4f76-93ba-7fd12f893c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:15.846519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841bd23-f5d1-4f76-93ba-7fd12f893c7c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:15.846832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841bd23-f5d1-4f76-93ba-7fd12f893c7c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:15.847105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841bd23-f5d1-4f76-93ba-7fd12f893c7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:15.848333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841bd23-f5d1-4f76-93ba-7fd12f893c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:15.849761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841bd23-f5d1-4f76-93ba-7fd12f893c7c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:15.851428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 571/1137] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:43:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:15.857686 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51b032-a7c7-4aa0-891e-4abdb99373b7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:15.970558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51b032-a7c7-4aa0-891e-4abdb99373b7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:16.005835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3413a-4c31-45d5-a7a1-2680bc199cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:16.006498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3413a-4c31-45d5-a7a1-2680bc199cda None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:16.006803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3413a-4c31-45d5-a7a1-2680bc199cda None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:16.007076 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3413a-4c31-45d5-a7a1-2680bc199cda None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:16.007786 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3413a-4c31-45d5-a7a1-2680bc199cda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:16.009207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3413a-4c31-45d5-a7a1-2680bc199cda None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:16.031449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 567/1138] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:43:15 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:43:18.364076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b7a81-cebf-45b8-9741-1d7e335dc58b None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:18.364607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b7a81-cebf-45b8-9741-1d7e335dc58b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:18.364607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b7a81-cebf-45b8-9741-1d7e335dc58b None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:18.366223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 572/1139] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:18.372351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2e78c-2ee5-41fc-a61e-8a2df85e7e64 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:18.372867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2e78c-2ee5-41fc-a61e-8a2df85e7e64 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:18.372974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2e78c-2ee5-41fc-a61e-8a2df85e7e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:18.390799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a6d2e78c-2ee5-41fc-a61e-8a2df85e7e64 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:18.391998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a6d2e78c-2ee5-41fc-a61e-8a2df85e7e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:18.462344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 568/1140] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:43:18.492591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b7a81-cebf-45b8-9741-1d7e335dc58b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:18.606685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b7a81-cebf-45b8-9741-1d7e335dc58b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:18.641788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90cf43-d916-4183-9e91-1dff0eb9108b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:18.642494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90cf43-d916-4183-9e91-1dff0eb9108b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:18.642795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90cf43-d916-4183-9e91-1dff0eb9108b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:18.643067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90cf43-d916-4183-9e91-1dff0eb9108b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:18.653610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90cf43-d916-4183-9e91-1dff0eb9108b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-cb90cf43-d916-4183-9e91-1dff0eb9108b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:43:18.661904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:43:18.663241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:43:18.664197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 573/1141] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:43:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:43:18.669539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2e78c-2ee5-41fc-a61e-8a2df85e7e64 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:18.782592 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2e78c-2ee5-41fc-a61e-8a2df85e7e64 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:18.830268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5efda0-f5a5-4378-8982-0eb8ef2a0dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:18.831295 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5efda0-f5a5-4378-8982-0eb8ef2a0dcf None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:18.831714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5efda0-f5a5-4378-8982-0eb8ef2a0dcf None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:18.832078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5efda0-f5a5-4378-8982-0eb8ef2a0dcf None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:18.833209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5efda0-f5a5-4378-8982-0eb8ef2a0dcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:18.833687 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5efda0-f5a5-4378-8982-0eb8ef2a0dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:18.835371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5efda0-f5a5-4378-8982-0eb8ef2a0dcf None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:18.846559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 569/1142] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:43:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:43:18.852384 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90cf43-d916-4183-9e91-1dff0eb9108b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:18.968891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90cf43-d916-4183-9e91-1dff0eb9108b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:19.004651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc2273-c470-4eda-8cfc-c917427e25cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:19.005530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc2273-c470-4eda-8cfc-c917427e25cb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:19.005805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc2273-c470-4eda-8cfc-c917427e25cb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:19.006078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc2273-c470-4eda-8cfc-c917427e25cb None admin] PATH_INFO: `/v3/users/090baabb50134fe6bf8796d678f31482` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:19.020109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc2273-c470-4eda-8cfc-c917427e25cb None admin] RBAC: Authorizing `identity:get_user(user_id=090baabb50134fe6bf8796d678f31482)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:19.021585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc2273-c470-4eda-8cfc-c917427e25cb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:19.023411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 574/1143] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:18 2022] GET /identity/v3/users/090baabb50134fe6bf8796d678f31482 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:19.033264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5efda0-f5a5-4378-8982-0eb8ef2a0dcf None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:19.149799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5efda0-f5a5-4378-8982-0eb8ef2a0dcf None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:19.189937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a3e71-36c8-4428-a0ec-e32482e298f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:19.190944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a3e71-36c8-4428-a0ec-e32482e298f1 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:19.191363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a3e71-36c8-4428-a0ec-e32482e298f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:19.191778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a3e71-36c8-4428-a0ec-e32482e298f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:19.269870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a3e71-36c8-4428-a0ec-e32482e298f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:19.271311 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a3e71-36c8-4428-a0ec-e32482e298f1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:19.374731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 570/1144] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:43:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:43:19.380906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc2273-c470-4eda-8cfc-c917427e25cb None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:19.481182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc2273-c470-4eda-8cfc-c917427e25cb None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:19.518768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffef0f9-4a17-487f-a43d-1ef87d76eef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:19.519644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffef0f9-4a17-487f-a43d-1ef87d76eef5 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:19.519980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffef0f9-4a17-487f-a43d-1ef87d76eef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:19.520194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffef0f9-4a17-487f-a43d-1ef87d76eef5 None admin] PATH_INFO: `/v3/projects/bb03268316a3480cb2ef78666c388262` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:19.521647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffef0f9-4a17-487f-a43d-1ef87d76eef5 None admin] RBAC: Authorizing `identity:get_project(project_id=bb03268316a3480cb2ef78666c388262)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:19.523251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffef0f9-4a17-487f-a43d-1ef87d76eef5 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:19.525124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 575/1145] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:43:19 2022] GET /identity/v3/projects/bb03268316a3480cb2ef78666c388262 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:19.532018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a3e71-36c8-4428-a0ec-e32482e298f1 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:19.635530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a3e71-36c8-4428-a0ec-e32482e298f1 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:19.673358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a561b-2616-4804-91e5-e2323e538276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:19.674008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a561b-2616-4804-91e5-e2323e538276 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:19.674293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a561b-2616-4804-91e5-e2323e538276 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:19.674558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a561b-2616-4804-91e5-e2323e538276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:19.675412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a561b-2616-4804-91e5-e2323e538276 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=bb03268316a3480cb2ef78666c388262, user.id=090baabb50134fe6bf8796d678f31482) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:19.675811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a561b-2616-4804-91e5-e2323e538276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:19.677289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a561b-2616-4804-91e5-e2323e538276 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:19.683847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 571/1146] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:43:19 2022] GET /identity/v3/role_assignments?user.id=090baabb50134fe6bf8796d678f31482&scope.project.id=bb03268316a3480cb2ef78666c388262&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:21.951467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae21c2-692e-4428-9a4a-86d706977040 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:21.952036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae21c2-692e-4428-9a4a-86d706977040 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:21.952036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae21c2-692e-4428-9a4a-86d706977040 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:21.953976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 576/1147] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:43:21.959905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a09d8-774a-4e75-a588-4431e43e0bd0 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:21.960424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a09d8-774a-4e75-a588-4431e43e0bd0 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:21.960526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a09d8-774a-4e75-a588-4431e43e0bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:21.979842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7d9a09d8-774a-4e75-a588-4431e43e0bd0 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:21.981071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7d9a09d8-774a-4e75-a588-4431e43e0bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:22.050469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 572/1148] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:43:22.074247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae21c2-692e-4428-9a4a-86d706977040 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:22.184321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae21c2-692e-4428-9a4a-86d706977040 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:22.226497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4acfb-c1ad-4ff1-82fe-1a47cc702fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:22.227243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4acfb-c1ad-4ff1-82fe-1a47cc702fae None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:22.227520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4acfb-c1ad-4ff1-82fe-1a47cc702fae None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:22.227789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4acfb-c1ad-4ff1-82fe-1a47cc702fae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:22.238700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4acfb-c1ad-4ff1-82fe-1a47cc702fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-cfa4acfb-c1ad-4ff1-82fe-1a47cc702fae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:43:22.246546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:43:22.247750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:43:22.249426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 577/1149] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:43:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:43:22.259242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a09d8-774a-4e75-a588-4431e43e0bd0 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:22.362467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a09d8-774a-4e75-a588-4431e43e0bd0 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:22.397548 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7444f-feef-453c-8099-925a4e4afccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:22.398210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7444f-feef-453c-8099-925a4e4afccf None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:22.398599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7444f-feef-453c-8099-925a4e4afccf None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:22.398799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7444f-feef-453c-8099-925a4e4afccf None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:22.399508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7444f-feef-453c-8099-925a4e4afccf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:22.399808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7444f-feef-453c-8099-925a4e4afccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:22.401186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7444f-feef-453c-8099-925a4e4afccf None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:22.411158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 573/1150] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:43:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:22.417434 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4acfb-c1ad-4ff1-82fe-1a47cc702fae None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:22.528868 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4acfb-c1ad-4ff1-82fe-1a47cc702fae None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:22.564333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c4995-cb79-4293-94a6-88a5d30f847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:22.565252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c4995-cb79-4293-94a6-88a5d30f847b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:22.565528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c4995-cb79-4293-94a6-88a5d30f847b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:22.565801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c4995-cb79-4293-94a6-88a5d30f847b None admin] PATH_INFO: `/v3/users/090baabb50134fe6bf8796d678f31482` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:22.567032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c4995-cb79-4293-94a6-88a5d30f847b None admin] RBAC: Authorizing `identity:get_user(user_id=090baabb50134fe6bf8796d678f31482)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:22.568469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c4995-cb79-4293-94a6-88a5d30f847b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:22.570106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 578/1151] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:22 2022] GET /identity/v3/users/090baabb50134fe6bf8796d678f31482 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:22.576314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7444f-feef-453c-8099-925a4e4afccf None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:22.682478 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7444f-feef-453c-8099-925a4e4afccf None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:22.716567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d22f0-3614-454a-b24d-8c10d1944c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:22.717425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d22f0-3614-454a-b24d-8c10d1944c52 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:22.717703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d22f0-3614-454a-b24d-8c10d1944c52 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:22.718027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d22f0-3614-454a-b24d-8c10d1944c52 None admin] PATH_INFO: `/v3/projects/bb03268316a3480cb2ef78666c388262` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:22.719240 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d22f0-3614-454a-b24d-8c10d1944c52 None admin] RBAC: Authorizing `identity:get_project(project_id=bb03268316a3480cb2ef78666c388262)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:22.720679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d22f0-3614-454a-b24d-8c10d1944c52 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:22.722398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 574/1152] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:43:22 2022] GET /identity/v3/projects/bb03268316a3480cb2ef78666c388262 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:22.728853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c4995-cb79-4293-94a6-88a5d30f847b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:22.829792 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c4995-cb79-4293-94a6-88a5d30f847b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:22.862074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98b79d-bd89-4af3-ac35-0b4b371abc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:22.862994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98b79d-bd89-4af3-ac35-0b4b371abc77 None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:22.863267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98b79d-bd89-4af3-ac35-0b4b371abc77 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:22.863537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98b79d-bd89-4af3-ac35-0b4b371abc77 None admin] PATH_INFO: `/v3/projects/bb03268316a3480cb2ef78666c388262/users/090baabb50134fe6bf8796d678f31482/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:22.865485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98b79d-bd89-4af3-ac35-0b4b371abc77 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb03268316a3480cb2ef78666c388262, user_id=090baabb50134fe6bf8796d678f31482, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:22.866955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98b79d-bd89-4af3-ac35-0b4b371abc77 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:22.880887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 579/1153] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:43:22 2022] PUT /identity/v3/projects/bb03268316a3480cb2ef78666c388262/users/090baabb50134fe6bf8796d678f31482/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:43:25.136774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32607fe2-fddd-4364-b7a7-af03540b9a69 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:25.137346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32607fe2-fddd-4364-b7a7-af03540b9a69 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:25.137346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32607fe2-fddd-4364-b7a7-af03540b9a69 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:25.138699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 575/1154] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:25.145117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf4c2b-ea62-4225-8a09-fa3ae0125fe6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:25.145624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf4c2b-ea62-4225-8a09-fa3ae0125fe6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:25.145755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf4c2b-ea62-4225-8a09-fa3ae0125fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:25.168186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1adf4c2b-ea62-4225-8a09-fa3ae0125fe6 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:25.169478 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1adf4c2b-ea62-4225-8a09-fa3ae0125fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:25.250212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 580/1155] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:43:25.273700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32607fe2-fddd-4364-b7a7-af03540b9a69 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:25.378184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32607fe2-fddd-4364-b7a7-af03540b9a69 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:25.416057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708c191-83ee-4b94-9f9a-38fa84165c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:25.417001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708c191-83ee-4b94-9f9a-38fa84165c55 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:25.417276 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708c191-83ee-4b94-9f9a-38fa84165c55 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:25.417541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708c191-83ee-4b94-9f9a-38fa84165c55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:25.428861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708c191-83ee-4b94-9f9a-38fa84165c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-8708c191-83ee-4b94-9f9a-38fa84165c55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:43:25.436492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:43:25.438130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:43:25.439303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 576/1156] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:43:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:43:25.444521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf4c2b-ea62-4225-8a09-fa3ae0125fe6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:25.546286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf4c2b-ea62-4225-8a09-fa3ae0125fe6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:25.580798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bde023-1666-4fcb-b5fd-65e83cd4fbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:25.581570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bde023-1666-4fcb-b5fd-65e83cd4fbdf None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:25.581851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bde023-1666-4fcb-b5fd-65e83cd4fbdf None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:25.582224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bde023-1666-4fcb-b5fd-65e83cd4fbdf None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:25.583015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bde023-1666-4fcb-b5fd-65e83cd4fbdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:25.583320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bde023-1666-4fcb-b5fd-65e83cd4fbdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:25.584714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bde023-1666-4fcb-b5fd-65e83cd4fbdf None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:25.594125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 581/1157] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:43:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:25.600002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708c191-83ee-4b94-9f9a-38fa84165c55 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:25.698811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708c191-83ee-4b94-9f9a-38fa84165c55 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:25.732518 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf88c9-eec2-47c4-b3ef-1ff86fea79ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:25.733420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf88c9-eec2-47c4-b3ef-1ff86fea79ed None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:25.733700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf88c9-eec2-47c4-b3ef-1ff86fea79ed None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:25.733971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf88c9-eec2-47c4-b3ef-1ff86fea79ed None admin] PATH_INFO: `/v3/users/090baabb50134fe6bf8796d678f31482` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:25.735235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf88c9-eec2-47c4-b3ef-1ff86fea79ed None admin] RBAC: Authorizing `identity:get_user(user_id=090baabb50134fe6bf8796d678f31482)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:25.736691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf88c9-eec2-47c4-b3ef-1ff86fea79ed None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:25.738335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 577/1158] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:25 2022] GET /identity/v3/users/090baabb50134fe6bf8796d678f31482 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:25.744138 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bde023-1666-4fcb-b5fd-65e83cd4fbdf None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:25.845615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bde023-1666-4fcb-b5fd-65e83cd4fbdf None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:25.878910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe2693-47e9-430e-9840-e4d5d3bb9eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:25.879560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe2693-47e9-430e-9840-e4d5d3bb9eeb None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:25.879841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe2693-47e9-430e-9840-e4d5d3bb9eeb None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:25.880116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe2693-47e9-430e-9840-e4d5d3bb9eeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:25.953324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe2693-47e9-430e-9840-e4d5d3bb9eeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:25.954820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe2693-47e9-430e-9840-e4d5d3bb9eeb None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:26.058010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 582/1159] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:43:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:43:26.065096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf88c9-eec2-47c4-b3ef-1ff86fea79ed None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:26.171569 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf88c9-eec2-47c4-b3ef-1ff86fea79ed None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:26.211927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c491df-ee32-40fc-884d-8851b4b181da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:26.212849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c491df-ee32-40fc-884d-8851b4b181da None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:26.213127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c491df-ee32-40fc-884d-8851b4b181da None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:26.213396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c491df-ee32-40fc-884d-8851b4b181da None admin] PATH_INFO: `/v3/projects/bb03268316a3480cb2ef78666c388262` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:26.214674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c491df-ee32-40fc-884d-8851b4b181da None admin] RBAC: Authorizing `identity:get_project(project_id=bb03268316a3480cb2ef78666c388262)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:26.217359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c491df-ee32-40fc-884d-8851b4b181da None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:26.220569 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 578/1160] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:43:26 2022] GET /identity/v3/projects/bb03268316a3480cb2ef78666c388262 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:43:26.228292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe2693-47e9-430e-9840-e4d5d3bb9eeb None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:26.334593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe2693-47e9-430e-9840-e4d5d3bb9eeb None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:26.378093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea84c79-a594-43c0-98d7-94be33a8078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:26.378778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea84c79-a594-43c0-98d7-94be33a8078c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:26.379060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea84c79-a594-43c0-98d7-94be33a8078c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:26.379331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea84c79-a594-43c0-98d7-94be33a8078c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:26.380170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea84c79-a594-43c0-98d7-94be33a8078c None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=bb03268316a3480cb2ef78666c388262, user.id=090baabb50134fe6bf8796d678f31482) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:26.380476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea84c79-a594-43c0-98d7-94be33a8078c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:26.381933 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea84c79-a594-43c0-98d7-94be33a8078c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:26.388557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 583/1161] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:43:26 2022] GET /identity/v3/role_assignments?user.id=090baabb50134fe6bf8796d678f31482&scope.project.id=bb03268316a3480cb2ef78666c388262&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:28.739095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4cf81-bd31-4b25-a5bd-9c83608df519 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:28.739691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4cf81-bd31-4b25-a5bd-9c83608df519 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:28.739691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4cf81-bd31-4b25-a5bd-9c83608df519 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:28.741028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 579/1162] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:28.748921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b539762-7398-4651-8c03-f2f86fc5cb24 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:28.749457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b539762-7398-4651-8c03-f2f86fc5cb24 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:28.749457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b539762-7398-4651-8c03-f2f86fc5cb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:28.767885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1b539762-7398-4651-8c03-f2f86fc5cb24 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:28.769135 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1b539762-7398-4651-8c03-f2f86fc5cb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:28.843667 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 584/1163] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:43:28.870108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4cf81-bd31-4b25-a5bd-9c83608df519 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:28.995122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4cf81-bd31-4b25-a5bd-9c83608df519 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:29.032774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7c1b4-8a0d-4d1a-96fe-3d2f540c85c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:29.033429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7c1b4-8a0d-4d1a-96fe-3d2f540c85c2 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:29.033715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7c1b4-8a0d-4d1a-96fe-3d2f540c85c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:29.033997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7c1b4-8a0d-4d1a-96fe-3d2f540c85c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:29.120432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7c1b4-8a0d-4d1a-96fe-3d2f540c85c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:29.121932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7c1b4-8a0d-4d1a-96fe-3d2f540c85c2 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:29.234804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 580/1164] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:43:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:43:29.241515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b539762-7398-4651-8c03-f2f86fc5cb24 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:29.347391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b539762-7398-4651-8c03-f2f86fc5cb24 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:29.389669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475d289-d4d6-43b3-8dee-fe3c1964ba26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:29.390468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475d289-d4d6-43b3-8dee-fe3c1964ba26 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:29.390762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475d289-d4d6-43b3-8dee-fe3c1964ba26 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:29.391034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475d289-d4d6-43b3-8dee-fe3c1964ba26 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:29.398214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475d289-d4d6-43b3-8dee-fe3c1964ba26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:29.399681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475d289-d4d6-43b3-8dee-fe3c1964ba26 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-3475d289-d4d6-43b3-8dee-fe3c1964ba26 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:43:29.414671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:43:29.416062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:43:29.417262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 585/1165] 10.222.0.79 () {56 vars in 1117 bytes} [Fri May 13 13:43:29 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:43:29.422704 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7c1b4-8a0d-4d1a-96fe-3d2f540c85c2 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:29.529333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7c1b4-8a0d-4d1a-96fe-3d2f540c85c2 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:29.568721 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc06d2-eb87-41b6-85c0-81820586235f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:29.569411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc06d2-eb87-41b6-85c0-81820586235f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:29.569693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc06d2-eb87-41b6-85c0-81820586235f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:29.569965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc06d2-eb87-41b6-85c0-81820586235f None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:29.570690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc06d2-eb87-41b6-85c0-81820586235f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:29.571025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc06d2-eb87-41b6-85c0-81820586235f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:29.572696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc06d2-eb87-41b6-85c0-81820586235f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:29.582479 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 581/1166] 10.222.0.79 () {56 vars in 1115 bytes} [Fri May 13 13:43:29 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:31.887638 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d9b89-001b-4a8e-a929-a6cb94cb10ab None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:31.888518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d9b89-001b-4a8e-a929-a6cb94cb10ab None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:31.888518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d9b89-001b-4a8e-a929-a6cb94cb10ab None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:31.890145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 586/1167] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:31.896236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d15eb-5344-433b-b4fe-76b2a572bcea None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:31.896797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d15eb-5344-433b-b4fe-76b2a572bcea None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:31.896968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d15eb-5344-433b-b4fe-76b2a572bcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:31.915779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-022d15eb-5344-433b-b4fe-76b2a572bcea None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:31.916995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-022d15eb-5344-433b-b4fe-76b2a572bcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:32.001767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 582/1168] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:43:32.025682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d9b89-001b-4a8e-a929-a6cb94cb10ab None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:32.158404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d9b89-001b-4a8e-a929-a6cb94cb10ab None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:32.195494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc29ce8-63fc-41ef-9252-4d8d777e1cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:32.196142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc29ce8-63fc-41ef-9252-4d8d777e1cbc None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:32.196683 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc29ce8-63fc-41ef-9252-4d8d777e1cbc None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:32.196952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc29ce8-63fc-41ef-9252-4d8d777e1cbc None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:32.197703 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc29ce8-63fc-41ef-9252-4d8d777e1cbc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:32.199122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc29ce8-63fc-41ef-9252-4d8d777e1cbc None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:32.215891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 587/1169] 10.222.0.79 () {60 vars in 1138 bytes} [Fri May 13 13:43:32 2022] POST /identity/v3/domains => generated 267 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:43:34.621285 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbf624-f591-434e-a70d-5462445f69cf None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:34.621874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbf624-f591-434e-a70d-5462445f69cf None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:34.621874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbf624-f591-434e-a70d-5462445f69cf None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:34.623843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 583/1170] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:43:34.630957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221cb2c-88b7-41d2-b7ce-ae5f9dc02375 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:34.631791 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221cb2c-88b7-41d2-b7ce-ae5f9dc02375 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:34.631791 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221cb2c-88b7-41d2-b7ce-ae5f9dc02375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:34.651560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e221cb2c-88b7-41d2-b7ce-ae5f9dc02375 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:34.652780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e221cb2c-88b7-41d2-b7ce-ae5f9dc02375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:34.734976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 588/1171] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:43:34.759676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbf624-f591-434e-a70d-5462445f69cf None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:34.876249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbf624-f591-434e-a70d-5462445f69cf None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:34.925282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70eb72-7ebb-4a4a-926a-f99abdeda2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:34.926162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70eb72-7ebb-4a4a-926a-f99abdeda2bc None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:34.926435 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70eb72-7ebb-4a4a-926a-f99abdeda2bc None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:34.926737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70eb72-7ebb-4a4a-926a-f99abdeda2bc None admin] PATH_INFO: `/v3/domains/b0a6fd825fd84543ba656efa5808d309` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:34.939877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70eb72-7ebb-4a4a-926a-f99abdeda2bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=b0a6fd825fd84543ba656efa5808d309)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:34.941372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70eb72-7ebb-4a4a-926a-f99abdeda2bc None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:34.943192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 584/1172] 10.222.0.79 () {56 vars in 1183 bytes} [Fri May 13 13:43:34 2022] GET /identity/v3/domains/b0a6fd825fd84543ba656efa5808d309 => generated 267 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:34.949646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221cb2c-88b7-41d2-b7ce-ae5f9dc02375 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:35.081673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221cb2c-88b7-41d2-b7ce-ae5f9dc02375 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:35.118663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb520c-b496-4fda-b202-5e82d7e605f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:35.119416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb520c-b496-4fda-b202-5e82d7e605f6 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:35.119621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb520c-b496-4fda-b202-5e82d7e605f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:35.119888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb520c-b496-4fda-b202-5e82d7e605f6 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:35.120696 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb520c-b496-4fda-b202-5e82d7e605f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:35.122122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb520c-b496-4fda-b202-5e82d7e605f6 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:35.148909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 589/1173] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:43:34 2022] POST /identity/v3/projects => generated 361 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:43:37.471297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1701ea8-b3a6-4507-bb86-1de483833b9d None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:37.472038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1701ea8-b3a6-4507-bb86-1de483833b9d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:37.472038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1701ea8-b3a6-4507-bb86-1de483833b9d None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:37.473233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 585/1174] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:43:37.479287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ffd26-0bd0-48fa-a749-d543016b2ac5 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:37.479891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ffd26-0bd0-48fa-a749-d543016b2ac5 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:37.479891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ffd26-0bd0-48fa-a749-d543016b2ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:37.496714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6c1ffd26-0bd0-48fa-a749-d543016b2ac5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:37.497906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6c1ffd26-0bd0-48fa-a749-d543016b2ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:37.567281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 590/1175] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:43:37.592972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1701ea8-b3a6-4507-bb86-1de483833b9d None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:37.705296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1701ea8-b3a6-4507-bb86-1de483833b9d None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:37.739848 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926984b-8ad1-4efa-b68f-60cb8de079f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:37.740633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926984b-8ad1-4efa-b68f-60cb8de079f1 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:37.740908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926984b-8ad1-4efa-b68f-60cb8de079f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:37.741180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926984b-8ad1-4efa-b68f-60cb8de079f1 None admin] PATH_INFO: `/v3/domains/b0a6fd825fd84543ba656efa5808d309` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:37.742389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926984b-8ad1-4efa-b68f-60cb8de079f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b0a6fd825fd84543ba656efa5808d309)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:37.743889 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926984b-8ad1-4efa-b68f-60cb8de079f1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:37.745543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 586/1176] 10.222.0.79 () {56 vars in 1183 bytes} [Fri May 13 13:43:37 2022] GET /identity/v3/domains/b0a6fd825fd84543ba656efa5808d309 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:37.751603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ffd26-0bd0-48fa-a749-d543016b2ac5 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:37.862518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ffd26-0bd0-48fa-a749-d543016b2ac5 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:37.900281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d6a31-d6b8-43d0-9974-aa07c5a6a1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:37.900948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d6a31-d6b8-43d0-9974-aa07c5a6a1b7 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:37.901227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d6a31-d6b8-43d0-9974-aa07c5a6a1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:37.901539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d6a31-d6b8-43d0-9974-aa07c5a6a1b7 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:37.902326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d6a31-d6b8-43d0-9974-aa07c5a6a1b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:37.903688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d6a31-d6b8-43d0-9974-aa07c5a6a1b7 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:37.927590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 591/1177] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:43:37 2022] POST /identity/v3/users => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:43:40.365928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932668b-6ac7-45dc-873e-3c93a6e1cc70 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:40.366459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932668b-6ac7-45dc-873e-3c93a6e1cc70 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:40.366459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932668b-6ac7-45dc-873e-3c93a6e1cc70 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:40.368067 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 587/1178] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:43:40.376664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba081f0f-5161-4b0a-a78a-25a8369b1dab None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:40.377218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba081f0f-5161-4b0a-a78a-25a8369b1dab None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:40.377218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba081f0f-5161-4b0a-a78a-25a8369b1dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:40.394430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-ba081f0f-5161-4b0a-a78a-25a8369b1dab None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:40.395670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ba081f0f-5161-4b0a-a78a-25a8369b1dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:40.461973 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 592/1179] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:43:40.486210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932668b-6ac7-45dc-873e-3c93a6e1cc70 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:40.589699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932668b-6ac7-45dc-873e-3c93a6e1cc70 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:40.622460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd51fce-2621-4bda-98a3-99b27b482b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:40.623726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51fce-2621-4bda-98a3-99b27b482b64 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:40.624009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51fce-2621-4bda-98a3-99b27b482b64 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:40.624271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51fce-2621-4bda-98a3-99b27b482b64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:40.634907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd51fce-2621-4bda-98a3-99b27b482b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-2cd51fce-2621-4bda-98a3-99b27b482b64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:43:40.643112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:43:40.644302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:43:40.650348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 588/1180] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:43:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:43:40.651285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba081f0f-5161-4b0a-a78a-25a8369b1dab None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:40.749465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba081f0f-5161-4b0a-a78a-25a8369b1dab None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:40.782488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158f9c9-f785-4633-b594-1009b7d5637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:40.783182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158f9c9-f785-4633-b594-1009b7d5637c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:40.783457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158f9c9-f785-4633-b594-1009b7d5637c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:40.783729 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158f9c9-f785-4633-b594-1009b7d5637c None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:40.784468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158f9c9-f785-4633-b594-1009b7d5637c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:40.784763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158f9c9-f785-4633-b594-1009b7d5637c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:40.786137 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158f9c9-f785-4633-b594-1009b7d5637c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:40.794407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 593/1181] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:43:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:40.800560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd51fce-2621-4bda-98a3-99b27b482b64 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:40.903636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd51fce-2621-4bda-98a3-99b27b482b64 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:40.944329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137dbc3-17e9-494a-8e6a-665f11d93fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:40.945202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137dbc3-17e9-494a-8e6a-665f11d93fff None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:40.945483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137dbc3-17e9-494a-8e6a-665f11d93fff None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:40.945757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137dbc3-17e9-494a-8e6a-665f11d93fff None admin] PATH_INFO: `/v3/users/bfacc776b9ba49d699f10bddf3388d40` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:40.961422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137dbc3-17e9-494a-8e6a-665f11d93fff None admin] RBAC: Authorizing `identity:get_user(user_id=bfacc776b9ba49d699f10bddf3388d40)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:40.962925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137dbc3-17e9-494a-8e6a-665f11d93fff None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:40.964687 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 589/1182] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:40 2022] GET /identity/v3/users/bfacc776b9ba49d699f10bddf3388d40 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:40.970819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158f9c9-f785-4633-b594-1009b7d5637c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:41.082592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158f9c9-f785-4633-b594-1009b7d5637c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:41.116662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbfb13-12c2-40f4-b4cb-d7dfd4e917d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:41.117314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbfb13-12c2-40f4-b4cb-d7dfd4e917d8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:41.117593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbfb13-12c2-40f4-b4cb-d7dfd4e917d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:41.117858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbfb13-12c2-40f4-b4cb-d7dfd4e917d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:41.184295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbfb13-12c2-40f4-b4cb-d7dfd4e917d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:41.186451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbfb13-12c2-40f4-b4cb-d7dfd4e917d8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:41.283588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 594/1183] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:43:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:43:41.289650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137dbc3-17e9-494a-8e6a-665f11d93fff None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:41.408690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137dbc3-17e9-494a-8e6a-665f11d93fff None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:41.459529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3096ef9-8afc-4cab-b4c4-f2598115d7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:41.460314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3096ef9-8afc-4cab-b4c4-f2598115d7b8 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:41.460587 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3096ef9-8afc-4cab-b4c4-f2598115d7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:41.460854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3096ef9-8afc-4cab-b4c4-f2598115d7b8 None admin] PATH_INFO: `/v3/projects/1f71c33db79b426daa7be367eb3ff8d3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:41.462056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3096ef9-8afc-4cab-b4c4-f2598115d7b8 None admin] RBAC: Authorizing `identity:get_project(project_id=1f71c33db79b426daa7be367eb3ff8d3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:41.463534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3096ef9-8afc-4cab-b4c4-f2598115d7b8 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:41.465271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 590/1184] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:43:41 2022] GET /identity/v3/projects/1f71c33db79b426daa7be367eb3ff8d3 => generated 361 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:41.472034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbfb13-12c2-40f4-b4cb-d7dfd4e917d8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:41.594633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbfb13-12c2-40f4-b4cb-d7dfd4e917d8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:41.629455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9848b2f-48c3-408b-93ef-f3ea4f126fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:41.630098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9848b2f-48c3-408b-93ef-f3ea4f126fa0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:41.630374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9848b2f-48c3-408b-93ef-f3ea4f126fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:41.630642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9848b2f-48c3-408b-93ef-f3ea4f126fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:41.631518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9848b2f-48c3-408b-93ef-f3ea4f126fa0 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=1f71c33db79b426daa7be367eb3ff8d3, user.id=bfacc776b9ba49d699f10bddf3388d40) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:41.631828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9848b2f-48c3-408b-93ef-f3ea4f126fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:41.633244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9848b2f-48c3-408b-93ef-f3ea4f126fa0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:41.639748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 595/1185] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:43:41 2022] GET /identity/v3/role_assignments?user.id=bfacc776b9ba49d699f10bddf3388d40&scope.project.id=1f71c33db79b426daa7be367eb3ff8d3&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:43.997640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e6c1b-66ef-4b59-8232-88ce71471c71 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:43.998211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e6c1b-66ef-4b59-8232-88ce71471c71 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:43.998315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e6c1b-66ef-4b59-8232-88ce71471c71 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:44.000186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 591/1186] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:44.006028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c22350-c773-407b-be0b-1144a2b724ac None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:44.006659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c22350-c773-407b-be0b-1144a2b724ac None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:44.006659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c22350-c773-407b-be0b-1144a2b724ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:44.023516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-75c22350-c773-407b-be0b-1144a2b724ac None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:44.024753 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-75c22350-c773-407b-be0b-1144a2b724ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:44.094228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 596/1187] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:43:44.117747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e6c1b-66ef-4b59-8232-88ce71471c71 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:44.221891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e6c1b-66ef-4b59-8232-88ce71471c71 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:44.259589 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae50f5-0b13-4137-8770-7d4b50d31e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:44.260299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae50f5-0b13-4137-8770-7d4b50d31e78 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:44.260581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae50f5-0b13-4137-8770-7d4b50d31e78 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:44.260860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae50f5-0b13-4137-8770-7d4b50d31e78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:44.272110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae50f5-0b13-4137-8770-7d4b50d31e78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-ceae50f5-0b13-4137-8770-7d4b50d31e78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:43:44.280484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:43:44.281740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:43:44.282909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 592/1188] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:43:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:43:44.287935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c22350-c773-407b-be0b-1144a2b724ac None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:44.416459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c22350-c773-407b-be0b-1144a2b724ac None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:44.450263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffb573-25d4-400b-87b4-7bd0cbabbd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:44.450987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffb573-25d4-400b-87b4-7bd0cbabbd27 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:44.451271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffb573-25d4-400b-87b4-7bd0cbabbd27 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:44.451547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffb573-25d4-400b-87b4-7bd0cbabbd27 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:44.452294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffb573-25d4-400b-87b4-7bd0cbabbd27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:44.452596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffb573-25d4-400b-87b4-7bd0cbabbd27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:44.453968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffb573-25d4-400b-87b4-7bd0cbabbd27 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:44.462281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 597/1189] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:43:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:44.469031 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae50f5-0b13-4137-8770-7d4b50d31e78 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:44.578358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae50f5-0b13-4137-8770-7d4b50d31e78 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:44.612106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845178c2-26b4-4217-9758-9d6ea083ae74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:44.612980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845178c2-26b4-4217-9758-9d6ea083ae74 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:44.613281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845178c2-26b4-4217-9758-9d6ea083ae74 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:44.613533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845178c2-26b4-4217-9758-9d6ea083ae74 None admin] PATH_INFO: `/v3/users/bfacc776b9ba49d699f10bddf3388d40` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:44.614737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845178c2-26b4-4217-9758-9d6ea083ae74 None admin] RBAC: Authorizing `identity:get_user(user_id=bfacc776b9ba49d699f10bddf3388d40)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:44.616185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845178c2-26b4-4217-9758-9d6ea083ae74 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:44.617821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 593/1190] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:44 2022] GET /identity/v3/users/bfacc776b9ba49d699f10bddf3388d40 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:44.624081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffb573-25d4-400b-87b4-7bd0cbabbd27 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:44.721745 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffb573-25d4-400b-87b4-7bd0cbabbd27 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:44.761466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe67ae-4dd1-48ca-98b7-03c75f417d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:44.762325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe67ae-4dd1-48ca-98b7-03c75f417d27 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:44.762605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe67ae-4dd1-48ca-98b7-03c75f417d27 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:44.762906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe67ae-4dd1-48ca-98b7-03c75f417d27 None admin] PATH_INFO: `/v3/projects/1f71c33db79b426daa7be367eb3ff8d3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:44.764114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe67ae-4dd1-48ca-98b7-03c75f417d27 None admin] RBAC: Authorizing `identity:get_project(project_id=1f71c33db79b426daa7be367eb3ff8d3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:44.765520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe67ae-4dd1-48ca-98b7-03c75f417d27 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:44.767271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 598/1191] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:43:44 2022] GET /identity/v3/projects/1f71c33db79b426daa7be367eb3ff8d3 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:44.773095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845178c2-26b4-4217-9758-9d6ea083ae74 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:44.880780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845178c2-26b4-4217-9758-9d6ea083ae74 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:44.914936 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044aa44-6f83-432e-962b-9de26812ce9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:44.915828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044aa44-6f83-432e-962b-9de26812ce9d None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:44.916104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044aa44-6f83-432e-962b-9de26812ce9d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:44.916379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044aa44-6f83-432e-962b-9de26812ce9d None admin] PATH_INFO: `/v3/projects/1f71c33db79b426daa7be367eb3ff8d3/users/bfacc776b9ba49d699f10bddf3388d40/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:44.918347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044aa44-6f83-432e-962b-9de26812ce9d None admin] RBAC: Authorizing `identity:create_grant(project_id=1f71c33db79b426daa7be367eb3ff8d3, user_id=bfacc776b9ba49d699f10bddf3388d40, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:44.920367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044aa44-6f83-432e-962b-9de26812ce9d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:44.933289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 594/1192] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:43:44 2022] PUT /identity/v3/projects/1f71c33db79b426daa7be367eb3ff8d3/users/bfacc776b9ba49d699f10bddf3388d40/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:43:47.173764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1ee0a-d56d-4e04-9635-a3e14dda83f4 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:47.174321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1ee0a-d56d-4e04-9635-a3e14dda83f4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:47.174321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1ee0a-d56d-4e04-9635-a3e14dda83f4 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:47.175694 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 599/1193] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:43:47.181386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152d2e9-7d3f-4550-87d8-c3793acc20ab None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:47.181834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152d2e9-7d3f-4550-87d8-c3793acc20ab None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:47.181983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152d2e9-7d3f-4550-87d8-c3793acc20ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:47.199510 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7152d2e9-7d3f-4550-87d8-c3793acc20ab None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:43:47.200712 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7152d2e9-7d3f-4550-87d8-c3793acc20ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:43:47.268486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 595/1194] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:43:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:43:47.293313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1ee0a-d56d-4e04-9635-a3e14dda83f4 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:47.419282 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1ee0a-d56d-4e04-9635-a3e14dda83f4 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:47.470640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b18ba-0bac-4d86-82eb-3a17f0c07f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:47.471467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b18ba-0bac-4d86-82eb-3a17f0c07f58 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:47.471747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b18ba-0bac-4d86-82eb-3a17f0c07f58 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:47.472018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b18ba-0bac-4d86-82eb-3a17f0c07f58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:47.483743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b18ba-0bac-4d86-82eb-3a17f0c07f58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-016b18ba-0bac-4d86-82eb-3a17f0c07f58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:43:47.491426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:43:47.493273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:43:47.495292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 600/1195] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:43:47.500442 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152d2e9-7d3f-4550-87d8-c3793acc20ab None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:47.610506 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152d2e9-7d3f-4550-87d8-c3793acc20ab None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:47.643327 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e2363-384f-47ba-9bd7-0cecbf8278dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:47.643989 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e2363-384f-47ba-9bd7-0cecbf8278dc None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:47.644264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e2363-384f-47ba-9bd7-0cecbf8278dc None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:47.644536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e2363-384f-47ba-9bd7-0cecbf8278dc None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:47.645264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e2363-384f-47ba-9bd7-0cecbf8278dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:47.645561 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e2363-384f-47ba-9bd7-0cecbf8278dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:47.646935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e2363-384f-47ba-9bd7-0cecbf8278dc None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:47.655443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 596/1196] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:47.661186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b18ba-0bac-4d86-82eb-3a17f0c07f58 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:47.755197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b18ba-0bac-4d86-82eb-3a17f0c07f58 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:47.790219 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c64e4d-e687-4b17-98b9-b7f282a56877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:47.791131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c64e4d-e687-4b17-98b9-b7f282a56877 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:47.791396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c64e4d-e687-4b17-98b9-b7f282a56877 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:47.791653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c64e4d-e687-4b17-98b9-b7f282a56877 None admin] PATH_INFO: `/v3/users/bfacc776b9ba49d699f10bddf3388d40` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:47.793005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c64e4d-e687-4b17-98b9-b7f282a56877 None admin] RBAC: Authorizing `identity:get_user(user_id=bfacc776b9ba49d699f10bddf3388d40)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:47.794453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c64e4d-e687-4b17-98b9-b7f282a56877 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:47.796131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 601/1197] 10.222.0.79 () {56 vars in 1177 bytes} [Fri May 13 13:43:47 2022] GET /identity/v3/users/bfacc776b9ba49d699f10bddf3388d40 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:47.803387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e2363-384f-47ba-9bd7-0cecbf8278dc None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:47.911239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e2363-384f-47ba-9bd7-0cecbf8278dc None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:47.944732 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eee00f-f1cb-412e-b719-500dd5f1ef87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:47.945372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eee00f-f1cb-412e-b719-500dd5f1ef87 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:47.945651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eee00f-f1cb-412e-b719-500dd5f1ef87 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:47.946189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eee00f-f1cb-412e-b719-500dd5f1ef87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:48.015954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eee00f-f1cb-412e-b719-500dd5f1ef87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:48.017412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eee00f-f1cb-412e-b719-500dd5f1ef87 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:48.112754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 597/1198] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:43:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:43:48.118731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c64e4d-e687-4b17-98b9-b7f282a56877 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:48.236798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c64e4d-e687-4b17-98b9-b7f282a56877 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:48.273206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc7ce3-4790-43d3-ba25-265892177162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:48.274047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc7ce3-4790-43d3-ba25-265892177162 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:48.274321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc7ce3-4790-43d3-ba25-265892177162 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:48.274587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc7ce3-4790-43d3-ba25-265892177162 None admin] PATH_INFO: `/v3/projects/1f71c33db79b426daa7be367eb3ff8d3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:48.275768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc7ce3-4790-43d3-ba25-265892177162 None admin] RBAC: Authorizing `identity:get_project(project_id=1f71c33db79b426daa7be367eb3ff8d3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:48.277319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc7ce3-4790-43d3-ba25-265892177162 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:48.279142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 602/1199] 10.222.0.79 () {56 vars in 1186 bytes} [Fri May 13 13:43:48 2022] GET /identity/v3/projects/1f71c33db79b426daa7be367eb3ff8d3 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:43:48.285479 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eee00f-f1cb-412e-b719-500dd5f1ef87 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:43:48.380460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eee00f-f1cb-412e-b719-500dd5f1ef87 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:43:48.413431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bb784-623b-4eb4-99f0-8bf9e34447b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:43:48.414078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bb784-623b-4eb4-99f0-8bf9e34447b2 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:43:48.414356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bb784-623b-4eb4-99f0-8bf9e34447b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:43:48.414638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bb784-623b-4eb4-99f0-8bf9e34447b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:43:48.415481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bb784-623b-4eb4-99f0-8bf9e34447b2 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=1f71c33db79b426daa7be367eb3ff8d3, user.id=bfacc776b9ba49d699f10bddf3388d40) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:43:48.415789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bb784-623b-4eb4-99f0-8bf9e34447b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:43:48.417218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bb784-623b-4eb4-99f0-8bf9e34447b2 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:43:48.422813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 598/1200] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:43:48 2022] GET /identity/v3/role_assignments?user.id=bfacc776b9ba49d699f10bddf3388d40&scope.project.id=1f71c33db79b426daa7be367eb3ff8d3&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:44:51.816075 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec52583-2e53-4f61-b41d-d0b098b5ca36 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:51.816980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec52583-2e53-4f61-b41d-d0b098b5ca36 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:51.816980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec52583-2e53-4f61-b41d-d0b098b5ca36 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:51.823027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 603/1201] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:44:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:44:51.829965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eef6b-af01-46cf-bfe8-a37d3ccc70e9 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:51.830850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eef6b-af01-46cf-bfe8-a37d3ccc70e9 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:51.831071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eef6b-af01-46cf-bfe8-a37d3ccc70e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:51.856237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-848eef6b-af01-46cf-bfe8-a37d3ccc70e9 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:44:51.857596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-848eef6b-af01-46cf-bfe8-a37d3ccc70e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:44:51.971288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 599/1202] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:44:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:44:51.998757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec52583-2e53-4f61-b41d-d0b098b5ca36 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:52.193477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec52583-2e53-4f61-b41d-d0b098b5ca36 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:52.262545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165858c-7975-420d-8af2-22c8e8acc2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:52.263963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165858c-7975-420d-8af2-22c8e8acc2ce None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:52.264443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165858c-7975-420d-8af2-22c8e8acc2ce None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:52.264948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165858c-7975-420d-8af2-22c8e8acc2ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:52.282223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165858c-7975-420d-8af2-22c8e8acc2ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:52.285490 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165858c-7975-420d-8af2-22c8e8acc2ce None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-f165858c-7975-420d-8af2-22c8e8acc2ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:44:52.319977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:44:52.324366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:44:52.327416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:44:52.327416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:44:52.327416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:44:52.327416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:44:52.327416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 604/1203] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:44:52.332085 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848eef6b-af01-46cf-bfe8-a37d3ccc70e9 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:52.493245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848eef6b-af01-46cf-bfe8-a37d3ccc70e9 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:52.554940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42567fb-af08-46cd-a006-182db3dadc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:52.555594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42567fb-af08-46cd-a006-182db3dadc66 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:52.555910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42567fb-af08-46cd-a006-182db3dadc66 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:52.556152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42567fb-af08-46cd-a006-182db3dadc66 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:52.557476 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42567fb-af08-46cd-a006-182db3dadc66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:44:52.557727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42567fb-af08-46cd-a006-182db3dadc66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:52.559173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42567fb-af08-46cd-a006-182db3dadc66 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:52.578701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 600/1204] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:44:52.586689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165858c-7975-420d-8af2-22c8e8acc2ce None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:52.731664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165858c-7975-420d-8af2-22c8e8acc2ce None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:52.779142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c49b24-34f6-4e98-b5dd-e36d69545d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:52.779780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c49b24-34f6-4e98-b5dd-e36d69545d59 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:52.780065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c49b24-34f6-4e98-b5dd-e36d69545d59 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:52.780341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c49b24-34f6-4e98-b5dd-e36d69545d59 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:52.781155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c49b24-34f6-4e98-b5dd-e36d69545d59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:52.782687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c49b24-34f6-4e98-b5dd-e36d69545d59 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:52.808144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 605/1205] 10.222.0.79 () {60 vars in 1132 bytes} [Fri May 13 13:44:52 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:44:55.623544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837ea4f-a4d1-49d1-8260-acffd1dced85 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:55.624145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837ea4f-a4d1-49d1-8260-acffd1dced85 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:55.624368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837ea4f-a4d1-49d1-8260-acffd1dced85 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:55.627385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 601/1206] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:44:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:44:55.634179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23136ac-722c-4a69-b5b1-8dd088f192d5 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:55.635537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23136ac-722c-4a69-b5b1-8dd088f192d5 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:55.635537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23136ac-722c-4a69-b5b1-8dd088f192d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:55.654726 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a23136ac-722c-4a69-b5b1-8dd088f192d5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:44:55.656166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a23136ac-722c-4a69-b5b1-8dd088f192d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:44:55.747719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 606/1207] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:44:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:44:55.772654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837ea4f-a4d1-49d1-8260-acffd1dced85 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:55.949063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837ea4f-a4d1-49d1-8260-acffd1dced85 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:56.006665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec57e4b-4340-4c89-8652-aa026d448b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:56.008447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec57e4b-4340-4c89-8652-aa026d448b9e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:56.009153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec57e4b-4340-4c89-8652-aa026d448b9e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:56.009647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec57e4b-4340-4c89-8652-aa026d448b9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:56.031354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec57e4b-4340-4c89-8652-aa026d448b9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-eec57e4b-4340-4c89-8652-aa026d448b9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:44:56.044255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:44:56.045379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:44:56.047064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 602/1208] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:44:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:44:56.054202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23136ac-722c-4a69-b5b1-8dd088f192d5 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:56.217355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23136ac-722c-4a69-b5b1-8dd088f192d5 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:56.266694 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec087efd-cfed-448c-a48b-7c3880eb209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:56.267417 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec087efd-cfed-448c-a48b-7c3880eb209e None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:56.267693 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec087efd-cfed-448c-a48b-7c3880eb209e None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:56.267962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec087efd-cfed-448c-a48b-7c3880eb209e None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:56.268732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec087efd-cfed-448c-a48b-7c3880eb209e None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:44:56.269070 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec087efd-cfed-448c-a48b-7c3880eb209e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:56.270470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec087efd-cfed-448c-a48b-7c3880eb209e None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:56.284249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 607/1209] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:44:56.290572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec57e4b-4340-4c89-8652-aa026d448b9e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:56.439253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec57e4b-4340-4c89-8652-aa026d448b9e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:56.484319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a3a8e-696a-41ec-986a-0d12f7518b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:56.485701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a3a8e-696a-41ec-986a-0d12f7518b3d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:56.486010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a3a8e-696a-41ec-986a-0d12f7518b3d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:56.486283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a3a8e-696a-41ec-986a-0d12f7518b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:56.502400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90a3a8e-696a-41ec-986a-0d12f7518b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:56.505197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90a3a8e-696a-41ec-986a-0d12f7518b3d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-b90a3a8e-696a-41ec-986a-0d12f7518b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:44:56.529035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:44:56.530487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:44:56.531987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:44:56.531987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:44:56.531987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:44:56.531987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:44:56.531987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 603/1210] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:44:56.537429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec087efd-cfed-448c-a48b-7c3880eb209e None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:56.682427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec087efd-cfed-448c-a48b-7c3880eb209e None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:56.741423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9f0a1-bb87-4b46-b532-4c2b3fdf19f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:56.742602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9f0a1-bb87-4b46-b532-4c2b3fdf19f0 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:56.743233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9f0a1-bb87-4b46-b532-4c2b3fdf19f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:56.743781 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9f0a1-bb87-4b46-b532-4c2b3fdf19f0 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:56.744790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9f0a1-bb87-4b46-b532-4c2b3fdf19f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:44:56.745176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9f0a1-bb87-4b46-b532-4c2b3fdf19f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:56.747101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9f0a1-bb87-4b46-b532-4c2b3fdf19f0 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:56.766467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 608/1211] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:44:56.773804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a3a8e-696a-41ec-986a-0d12f7518b3d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:56.917505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a3a8e-696a-41ec-986a-0d12f7518b3d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:56.960861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbed0e1-7607-4a2d-92ed-91831215c07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:56.962037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbed0e1-7607-4a2d-92ed-91831215c07e None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:56.962364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbed0e1-7607-4a2d-92ed-91831215c07e None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:56.962798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbed0e1-7607-4a2d-92ed-91831215c07e None admin] PATH_INFO: `/v3/users/placement` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:56.976002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbed0e1-7607-4a2d-92ed-91831215c07e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:56.979318 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbed0e1-7607-4a2d-92ed-91831215c07e None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-ddbed0e1-7607-4a2d-92ed-91831215c07e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:44:57.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:44:57.002858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:44:57.004182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 604/1212] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:44:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:44:57.008850 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9f0a1-bb87-4b46-b532-4c2b3fdf19f0 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:57.145451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9f0a1-bb87-4b46-b532-4c2b3fdf19f0 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:57.194195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f62b-b9aa-4aa2-9dd6-ac4444623c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:57.195416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f62b-b9aa-4aa2-9dd6-ac4444623c02 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:57.195971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f62b-b9aa-4aa2-9dd6-ac4444623c02 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:57.196502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f62b-b9aa-4aa2-9dd6-ac4444623c02 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:57.198509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f62b-b9aa-4aa2-9dd6-ac4444623c02 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:44:57.199008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f62b-b9aa-4aa2-9dd6-ac4444623c02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:57.201260 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f62b-b9aa-4aa2-9dd6-ac4444623c02 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:57.228196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 609/1213] 10.222.0.79 () {56 vars in 1143 bytes} [Fri May 13 13:44:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:44:57.234806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbed0e1-7607-4a2d-92ed-91831215c07e None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:57.386757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbed0e1-7607-4a2d-92ed-91831215c07e None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:57.436512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de9a2f-ad50-49f8-8ec6-a686d5858ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:57.437262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de9a2f-ad50-49f8-8ec6-a686d5858ab3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:57.437509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de9a2f-ad50-49f8-8ec6-a686d5858ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:57.437781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de9a2f-ad50-49f8-8ec6-a686d5858ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:57.544926 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de9a2f-ad50-49f8-8ec6-a686d5858ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:57.546414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de9a2f-ad50-49f8-8ec6-a686d5858ab3 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:57.710076 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 605/1214] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:44:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:44:57.711864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f62b-b9aa-4aa2-9dd6-ac4444623c02 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:57.860646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f62b-b9aa-4aa2-9dd6-ac4444623c02 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:57.923943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0c95c-1719-4221-88b4-e94298047cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:57.925134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0c95c-1719-4221-88b4-e94298047cff None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:57.925640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0c95c-1719-4221-88b4-e94298047cff None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:57.925998 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0c95c-1719-4221-88b4-e94298047cff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:57.939833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0c95c-1719-4221-88b4-e94298047cff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:57.941309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0c95c-1719-4221-88b4-e94298047cff None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-f4c0c95c-1719-4221-88b4-e94298047cff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:44:57.967530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:44:57.971519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:44:57.972443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:44:57.972443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:44:57.972443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:44:57.972443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:44:57.972443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 610/1215] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:44:57.975685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de9a2f-ad50-49f8-8ec6-a686d5858ab3 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:58.130013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de9a2f-ad50-49f8-8ec6-a686d5858ab3 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:58.187980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e5348-9130-4bbb-a779-5ca786a39880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:58.188406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e5348-9130-4bbb-a779-5ca786a39880 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:58.188597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e5348-9130-4bbb-a779-5ca786a39880 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:58.188884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e5348-9130-4bbb-a779-5ca786a39880 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:58.189855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e5348-9130-4bbb-a779-5ca786a39880 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:44:58.190026 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e5348-9130-4bbb-a779-5ca786a39880 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:58.191613 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e5348-9130-4bbb-a779-5ca786a39880 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:58.215172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 606/1216] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:44:58.217452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0c95c-1719-4221-88b4-e94298047cff None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:58.349698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0c95c-1719-4221-88b4-e94298047cff None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:58.394045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5355d-f9ce-4a78-b92a-55888a751c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:58.394879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5355d-f9ce-4a78-b92a-55888a751c3a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:58.395163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5355d-f9ce-4a78-b92a-55888a751c3a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:58.395430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5355d-f9ce-4a78-b92a-55888a751c3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:58.406189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5355d-f9ce-4a78-b92a-55888a751c3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:58.407974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5355d-f9ce-4a78-b92a-55888a751c3a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-73c5355d-f9ce-4a78-b92a-55888a751c3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:44:58.425005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:44:58.426245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:44:58.427425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:44:58.427425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:44:58.427425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:44:58.427425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:44:58.427425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:44:58.427425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:44:58.427425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:44:58.427425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:44:58.429109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 611/1217] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:44:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:44:58.436410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e5348-9130-4bbb-a779-5ca786a39880 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:58.562941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e5348-9130-4bbb-a779-5ca786a39880 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:58.610189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053dcb9-026a-4428-b295-1dc4ea8a740f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:58.610803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053dcb9-026a-4428-b295-1dc4ea8a740f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:58.611081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053dcb9-026a-4428-b295-1dc4ea8a740f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:58.611349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053dcb9-026a-4428-b295-1dc4ea8a740f None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:58.612143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053dcb9-026a-4428-b295-1dc4ea8a740f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:44:58.612446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053dcb9-026a-4428-b295-1dc4ea8a740f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:58.613895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053dcb9-026a-4428-b295-1dc4ea8a740f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:58.628623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 607/1218] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:44:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:44:58.635348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5355d-f9ce-4a78-b92a-55888a751c3a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:44:58.803212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5355d-f9ce-4a78-b92a-55888a751c3a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:44:58.854828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b5a93-edc6-4594-9fc1-553b7fe0e91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:44:58.855923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5a93-edc6-4594-9fc1-553b7fe0e91e None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:44:58.856490 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5a93-edc6-4594-9fc1-553b7fe0e91e None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:44:58.856880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5a93-edc6-4594-9fc1-553b7fe0e91e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:44:58.858163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b5a93-edc6-4594-9fc1-553b7fe0e91e None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=4ae9a6f1ac304e21b8d3115f2a0baf88) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:44:58.858556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b5a93-edc6-4594-9fc1-553b7fe0e91e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:44:58.860299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b5a93-edc6-4594-9fc1-553b7fe0e91e None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:44:58.867112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 612/1219] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:44:58 2022] GET /identity/v3/role_assignments?user.id=4ae9a6f1ac304e21b8d3115f2a0baf88&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:45:01.501592 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49301b-f25e-49e7-ba35-4b91e8d546ab None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:01.502384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49301b-f25e-49e7-ba35-4b91e8d546ab None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:01.502384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49301b-f25e-49e7-ba35-4b91e8d546ab None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:01.504702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 608/1220] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:45:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:45:01.512477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb81a-9c20-45c7-967a-e7c49557349c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:01.512477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb81a-9c20-45c7-967a-e7c49557349c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:01.513907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb81a-9c20-45c7-967a-e7c49557349c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:01.535518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1cccb81a-9c20-45c7-967a-e7c49557349c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:45:01.536830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1cccb81a-9c20-45c7-967a-e7c49557349c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:45:01.630822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 613/1221] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:45:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:45:01.656347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49301b-f25e-49e7-ba35-4b91e8d546ab None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:01.791280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49301b-f25e-49e7-ba35-4b91e8d546ab None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:01.833444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea92301-7130-4ac0-9ce9-2c33375653fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:01.834291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92301-7130-4ac0-9ce9-2c33375653fe None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:01.834566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92301-7130-4ac0-9ce9-2c33375653fe None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:01.834925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92301-7130-4ac0-9ce9-2c33375653fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:01.847230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea92301-7130-4ac0-9ce9-2c33375653fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-8ea92301-7130-4ac0-9ce9-2c33375653fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:45:01.855901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:01.856941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:01.857785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 609/1222] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:45:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:01.864186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccb81a-9c20-45c7-967a-e7c49557349c None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:01.978190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccb81a-9c20-45c7-967a-e7c49557349c None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:02.026268 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080217f3-e993-4927-9245-b0816b2b271c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:02.027533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080217f3-e993-4927-9245-b0816b2b271c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:02.028107 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080217f3-e993-4927-9245-b0816b2b271c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:02.028522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080217f3-e993-4927-9245-b0816b2b271c None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:02.029441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080217f3-e993-4927-9245-b0816b2b271c None admin] RBAC: Adding query filter params (name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:02.029957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080217f3-e993-4927-9245-b0816b2b271c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:02.032084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080217f3-e993-4927-9245-b0816b2b271c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:02.051679 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 614/1223] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:45:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:02.052933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea92301-7130-4ac0-9ce9-2c33375653fe None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:02.180466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea92301-7130-4ac0-9ce9-2c33375653fe None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:02.221858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c3ed1-2c72-435f-99ff-72328242a444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:02.222640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c3ed1-2c72-435f-99ff-72328242a444 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:02.222942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c3ed1-2c72-435f-99ff-72328242a444 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:02.223207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c3ed1-2c72-435f-99ff-72328242a444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:02.234677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c3ed1-2c72-435f-99ff-72328242a444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:02.236362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c3ed1-2c72-435f-99ff-72328242a444 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-960c3ed1-2c72-435f-99ff-72328242a444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:45:02.258325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:02.261002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:02.262612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:45:02.262612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:45:02.262612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:02.262612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:02.262612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 610/1224] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:02.267296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080217f3-e993-4927-9245-b0816b2b271c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:02.422834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080217f3-e993-4927-9245-b0816b2b271c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:02.467055 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f5e03-47b6-4dc6-b72f-b31b33a1f80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:02.467735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5e03-47b6-4dc6-b72f-b31b33a1f80a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:02.468007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5e03-47b6-4dc6-b72f-b31b33a1f80a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:02.468283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5e03-47b6-4dc6-b72f-b31b33a1f80a None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:02.468998 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f5e03-47b6-4dc6-b72f-b31b33a1f80a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:02.469299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f5e03-47b6-4dc6-b72f-b31b33a1f80a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:02.470842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f5e03-47b6-4dc6-b72f-b31b33a1f80a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:02.484183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 615/1225] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:02.492565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c3ed1-2c72-435f-99ff-72328242a444 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:02.630675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c3ed1-2c72-435f-99ff-72328242a444 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:02.667755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abf3f9-2ad9-44ea-8bd0-7730d2acf013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:02.668699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abf3f9-2ad9-44ea-8bd0-7730d2acf013 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:02.669120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abf3f9-2ad9-44ea-8bd0-7730d2acf013 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:02.669446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abf3f9-2ad9-44ea-8bd0-7730d2acf013 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:02.680756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abf3f9-2ad9-44ea-8bd0-7730d2acf013 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:02.682429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abf3f9-2ad9-44ea-8bd0-7730d2acf013 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-34abf3f9-2ad9-44ea-8bd0-7730d2acf013 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:45:02.699743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:02.707581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:02.708859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 611/1226] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:02.710159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f5e03-47b6-4dc6-b72f-b31b33a1f80a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:02.829971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f5e03-47b6-4dc6-b72f-b31b33a1f80a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:02.871694 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff26a84-4821-41e0-8fbd-d563e8ebfbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:02.872367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff26a84-4821-41e0-8fbd-d563e8ebfbd2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:02.872654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff26a84-4821-41e0-8fbd-d563e8ebfbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:02.872994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff26a84-4821-41e0-8fbd-d563e8ebfbd2 None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:02.873920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff26a84-4821-41e0-8fbd-d563e8ebfbd2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:02.874227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff26a84-4821-41e0-8fbd-d563e8ebfbd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:02.875691 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff26a84-4821-41e0-8fbd-d563e8ebfbd2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:02.899840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 616/1227] 10.222.0.79 () {56 vars in 1143 bytes} [Fri May 13 13:45:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:45:02.908048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abf3f9-2ad9-44ea-8bd0-7730d2acf013 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:03.033806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abf3f9-2ad9-44ea-8bd0-7730d2acf013 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:03.077316 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7469e-67dd-468c-9f35-b1f134f56564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:03.078531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7469e-67dd-468c-9f35-b1f134f56564 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:03.079004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7469e-67dd-468c-9f35-b1f134f56564 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:03.079428 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7469e-67dd-468c-9f35-b1f134f56564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:03.091965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7469e-67dd-468c-9f35-b1f134f56564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:03.093425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa7469e-67dd-468c-9f35-b1f134f56564 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-bfa7469e-67dd-468c-9f35-b1f134f56564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:45:03.114607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:03.115871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:03.116793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:45:03.116793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:45:03.116793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:03.116793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:03.116793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 612/1228] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:03.121622 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff26a84-4821-41e0-8fbd-d563e8ebfbd2 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:03.233919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff26a84-4821-41e0-8fbd-d563e8ebfbd2 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:03.272474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ba403-26ba-4c9e-98b0-b7a1be5eb42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:03.273319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ba403-26ba-4c9e-98b0-b7a1be5eb42b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:03.273675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ba403-26ba-4c9e-98b0-b7a1be5eb42b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:03.273955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ba403-26ba-4c9e-98b0-b7a1be5eb42b None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:03.274833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ba403-26ba-4c9e-98b0-b7a1be5eb42b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:03.275176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ba403-26ba-4c9e-98b0-b7a1be5eb42b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:03.276723 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ba403-26ba-4c9e-98b0-b7a1be5eb42b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:03.289321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 617/1229] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:03.295682 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7469e-67dd-468c-9f35-b1f134f56564 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:03.417690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7469e-67dd-468c-9f35-b1f134f56564 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:03.464434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7bfea-a829-4714-9736-bf1d72126ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:03.465430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7bfea-a829-4714-9736-bf1d72126ca5 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:03.465711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7bfea-a829-4714-9736-bf1d72126ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:03.465987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7bfea-a829-4714-9736-bf1d72126ca5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:03.475144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7bfea-a829-4714-9736-bf1d72126ca5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:03.476945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7bfea-a829-4714-9736-bf1d72126ca5 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-87d7bfea-a829-4714-9736-bf1d72126ca5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:45:03.496968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:03.498183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:03.499262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:45:03.499262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:45:03.499262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:45:03.499262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:45:03.499262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:45:03.499262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:45:03.499262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:45:03.499262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:03.499262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 613/1230] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:45:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:03.506256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ba403-26ba-4c9e-98b0-b7a1be5eb42b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:03.648045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ba403-26ba-4c9e-98b0-b7a1be5eb42b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:03.691339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6002dc9-cb63-4a6f-a6e8-db31f1b58538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:03.692138 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6002dc9-cb63-4a6f-a6e8-db31f1b58538 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:03.692734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6002dc9-cb63-4a6f-a6e8-db31f1b58538 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:03.693080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6002dc9-cb63-4a6f-a6e8-db31f1b58538 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:03.694142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6002dc9-cb63-4a6f-a6e8-db31f1b58538 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:03.694276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6002dc9-cb63-4a6f-a6e8-db31f1b58538 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:03.695758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6002dc9-cb63-4a6f-a6e8-db31f1b58538 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:03.722163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 618/1231] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:45:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:45:03.729623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7bfea-a829-4714-9736-bf1d72126ca5 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:03.875403 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7bfea-a829-4714-9736-bf1d72126ca5 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:03.919395 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdfd45-9d07-4133-9ff7-e15e1a2af5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:03.920507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdfd45-9d07-4133-9ff7-e15e1a2af5c2 None admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:03.920900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdfd45-9d07-4133-9ff7-e15e1a2af5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:03.921332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdfd45-9d07-4133-9ff7-e15e1a2af5c2 None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/4ae9a6f1ac304e21b8d3115f2a0baf88/roles/879ba54490a944d6abcf2ac5e2f1bea9` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:03.938323 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdfd45-9d07-4133-9ff7-e15e1a2af5c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=4ae9a6f1ac304e21b8d3115f2a0baf88, role_id=879ba54490a944d6abcf2ac5e2f1bea9)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:03.939790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdfd45-9d07-4133-9ff7-e15e1a2af5c2 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:03.952564 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 614/1232] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:45:03 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/4ae9a6f1ac304e21b8d3115f2a0baf88/roles/879ba54490a944d6abcf2ac5e2f1bea9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:45:06.695974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5b7ce-8502-49d0-af68-68b2378734db None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:06.695974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5b7ce-8502-49d0-af68-68b2378734db None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:06.695974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5b7ce-8502-49d0-af68-68b2378734db None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:06.697787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 619/1233] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:45:06.704542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae1f1-157c-4ccc-aef2-2e9371753619 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:06.705094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae1f1-157c-4ccc-aef2-2e9371753619 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:06.705550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae1f1-157c-4ccc-aef2-2e9371753619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:06.725502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e54ae1f1-157c-4ccc-aef2-2e9371753619 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:45:06.726873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e54ae1f1-157c-4ccc-aef2-2e9371753619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:45:06.800884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 615/1234] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:45:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:45:06.826345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5b7ce-8502-49d0-af68-68b2378734db None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:06.944094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5b7ce-8502-49d0-af68-68b2378734db None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:06.987018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f10a9-28a2-414a-8fb3-35e64d304011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:06.987834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f10a9-28a2-414a-8fb3-35e64d304011 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:06.988113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f10a9-28a2-414a-8fb3-35e64d304011 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:06.988384 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f10a9-28a2-414a-8fb3-35e64d304011 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:07.002964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f10a9-28a2-414a-8fb3-35e64d304011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-1e8f10a9-28a2-414a-8fb3-35e64d304011 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:45:07.008085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:07.009932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:07.011177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 620/1235] 10.222.0.79 () {56 vars in 1102 bytes} [Fri May 13 13:45:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:45:07.016780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ae1f1-157c-4ccc-aef2-2e9371753619 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:07.153291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ae1f1-157c-4ccc-aef2-2e9371753619 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:07.199970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bbee6-babc-4ec6-89bf-37532af10553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:07.200955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bbee6-babc-4ec6-89bf-37532af10553 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:07.201461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bbee6-babc-4ec6-89bf-37532af10553 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:07.201958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bbee6-babc-4ec6-89bf-37532af10553 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:07.204043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bbee6-babc-4ec6-89bf-37532af10553 None admin] RBAC: Adding query filter params (name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:07.204233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bbee6-babc-4ec6-89bf-37532af10553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:07.206497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bbee6-babc-4ec6-89bf-37532af10553 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:07.221700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 616/1236] 10.222.0.79 () {56 vars in 1103 bytes} [Fri May 13 13:45:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:45:07.230929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f10a9-28a2-414a-8fb3-35e64d304011 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:07.381284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f10a9-28a2-414a-8fb3-35e64d304011 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:07.430755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac1032-fd61-43e5-bc53-2373f2f5cb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:07.431714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac1032-fd61-43e5-bc53-2373f2f5cb1d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:07.432066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac1032-fd61-43e5-bc53-2373f2f5cb1d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:07.432314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac1032-fd61-43e5-bc53-2373f2f5cb1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:07.449622 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac1032-fd61-43e5-bc53-2373f2f5cb1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:07.451439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac1032-fd61-43e5-bc53-2373f2f5cb1d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-d4ac1032-fd61-43e5-bc53-2373f2f5cb1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:45:07.476030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:07.477122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:07.484333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:45:07.484333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:45:07.484333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:07.484333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:07.484333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 621/1237] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:45:07.484703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bbee6-babc-4ec6-89bf-37532af10553 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:07.609976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bbee6-babc-4ec6-89bf-37532af10553 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:07.655027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698c4d6-1926-4faa-9119-1ab782f12252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:07.655834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698c4d6-1926-4faa-9119-1ab782f12252 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:07.656264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698c4d6-1926-4faa-9119-1ab782f12252 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:07.656541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698c4d6-1926-4faa-9119-1ab782f12252 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:07.657278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698c4d6-1926-4faa-9119-1ab782f12252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:07.657749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698c4d6-1926-4faa-9119-1ab782f12252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:07.659297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698c4d6-1926-4faa-9119-1ab782f12252 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:07.675031 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 617/1238] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:07.682406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac1032-fd61-43e5-bc53-2373f2f5cb1d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:07.807363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac1032-fd61-43e5-bc53-2373f2f5cb1d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:07.849324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45305ba4-81b9-4d6d-97bb-5c9263fcecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:07.850642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45305ba4-81b9-4d6d-97bb-5c9263fcecfd None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:07.851268 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45305ba4-81b9-4d6d-97bb-5c9263fcecfd None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:07.851828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45305ba4-81b9-4d6d-97bb-5c9263fcecfd None admin] PATH_INFO: `/v3/users/placement` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:07.866827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45305ba4-81b9-4d6d-97bb-5c9263fcecfd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:07.870205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45305ba4-81b9-4d6d-97bb-5c9263fcecfd None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-45305ba4-81b9-4d6d-97bb-5c9263fcecfd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:45:07.904554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:07.906453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:07.908379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 622/1239] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:45:07.917268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698c4d6-1926-4faa-9119-1ab782f12252 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:08.038312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698c4d6-1926-4faa-9119-1ab782f12252 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:08.084836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20d635-d56a-49ab-8ef7-cc24479a61f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:08.085882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d635-d56a-49ab-8ef7-cc24479a61f3 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:08.086302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d635-d56a-49ab-8ef7-cc24479a61f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:08.086817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20d635-d56a-49ab-8ef7-cc24479a61f3 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:08.088377 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20d635-d56a-49ab-8ef7-cc24479a61f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:08.088914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20d635-d56a-49ab-8ef7-cc24479a61f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:08.090929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20d635-d56a-49ab-8ef7-cc24479a61f3 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:08.121708 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 618/1240] 10.222.0.79 () {56 vars in 1143 bytes} [Fri May 13 13:45:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:08.127484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45305ba4-81b9-4d6d-97bb-5c9263fcecfd None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:08.278524 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45305ba4-81b9-4d6d-97bb-5c9263fcecfd None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:08.326144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8690df-b393-4e1c-a947-2b1d10e84f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:08.326877 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8690df-b393-4e1c-a947-2b1d10e84f4c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:08.327151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8690df-b393-4e1c-a947-2b1d10e84f4c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:08.327453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8690df-b393-4e1c-a947-2b1d10e84f4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:08.402461 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8690df-b393-4e1c-a947-2b1d10e84f4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:08.403887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8690df-b393-4e1c-a947-2b1d10e84f4c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:08.531210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 623/1241] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:45:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:45:08.537577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20d635-d56a-49ab-8ef7-cc24479a61f3 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:08.670980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20d635-d56a-49ab-8ef7-cc24479a61f3 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:08.715807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fbb42-a1c1-416d-a089-cbe83834627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:08.716610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fbb42-a1c1-416d-a089-cbe83834627d None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:08.716886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fbb42-a1c1-416d-a089-cbe83834627d None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:08.717156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fbb42-a1c1-416d-a089-cbe83834627d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:08.728305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fbb42-a1c1-416d-a089-cbe83834627d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:08.729928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fbb42-a1c1-416d-a089-cbe83834627d None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-826fbb42-a1c1-416d-a089-cbe83834627d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:45:08.749356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:08.751117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:08.752360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:45:08.752360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:45:08.752360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:08.752360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:08.756247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 619/1242] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:08.757477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8690df-b393-4e1c-a947-2b1d10e84f4c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:08.885325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8690df-b393-4e1c-a947-2b1d10e84f4c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:08.937582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf2ac3-e5e2-448b-bf07-71cec8bc84b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:08.938258 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf2ac3-e5e2-448b-bf07-71cec8bc84b8 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:08.938540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf2ac3-e5e2-448b-bf07-71cec8bc84b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:08.938845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf2ac3-e5e2-448b-bf07-71cec8bc84b8 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:08.939562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf2ac3-e5e2-448b-bf07-71cec8bc84b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:08.939870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf2ac3-e5e2-448b-bf07-71cec8bc84b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:08.941247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf2ac3-e5e2-448b-bf07-71cec8bc84b8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:08.961389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 624/1243] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:08.971667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fbb42-a1c1-416d-a089-cbe83834627d None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:09.105542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fbb42-a1c1-416d-a089-cbe83834627d None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:09.144223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a1149-9020-4466-bfe3-e57658ef48ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:09.145059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a1149-9020-4466-bfe3-e57658ef48ba None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:09.145342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a1149-9020-4466-bfe3-e57658ef48ba None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:09.145627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a1149-9020-4466-bfe3-e57658ef48ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:09.153815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a1149-9020-4466-bfe3-e57658ef48ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:09.155409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a1149-9020-4466-bfe3-e57658ef48ba None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-709a1149-9020-4466-bfe3-e57658ef48ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:45:09.172188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:09.174422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:09.176625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:45:09.176625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:45:09.176625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:45:09.176625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:45:09.176625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:45:09.176625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:45:09.176625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:45:09.176625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:09.176625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 620/1244] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:45:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:09.179582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf2ac3-e5e2-448b-bf07-71cec8bc84b8 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:09.302778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf2ac3-e5e2-448b-bf07-71cec8bc84b8 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:09.347481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dd4f0-a310-4879-9faa-10b180dfc609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:09.348219 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dd4f0-a310-4879-9faa-10b180dfc609 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:09.348609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dd4f0-a310-4879-9faa-10b180dfc609 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:09.348959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dd4f0-a310-4879-9faa-10b180dfc609 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:09.349758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dd4f0-a310-4879-9faa-10b180dfc609 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:09.350070 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dd4f0-a310-4879-9faa-10b180dfc609 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:09.351692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dd4f0-a310-4879-9faa-10b180dfc609 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:09.366969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 625/1245] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:45:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:09.374208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a1149-9020-4466-bfe3-e57658ef48ba None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:09.529229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a1149-9020-4466-bfe3-e57658ef48ba None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:09.571694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddcc2e-7911-4141-afaa-6d4a1590ecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:09.572392 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddcc2e-7911-4141-afaa-6d4a1590ecfd None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:09.572778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddcc2e-7911-4141-afaa-6d4a1590ecfd None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:09.573078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddcc2e-7911-4141-afaa-6d4a1590ecfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:09.573998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddcc2e-7911-4141-afaa-6d4a1590ecfd None admin] RBAC: Adding query filter params (role.id=879ba54490a944d6abcf2ac5e2f1bea9, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=4ae9a6f1ac304e21b8d3115f2a0baf88) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:09.574410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddcc2e-7911-4141-afaa-6d4a1590ecfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:09.576292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddcc2e-7911-4141-afaa-6d4a1590ecfd None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:09.595225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 621/1246] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:45:09 2022] GET /identity/v3/role_assignments?user.id=4ae9a6f1ac304e21b8d3115f2a0baf88&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=879ba54490a944d6abcf2ac5e2f1bea9 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:12.477329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f85310-ede4-4946-be31-65c325f934d7 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:12.477924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f85310-ede4-4946-be31-65c325f934d7 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:12.478066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f85310-ede4-4946-be31-65c325f934d7 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:12.479912 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 626/1247] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:45:12.489825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95368-17e3-49c5-a861-87ce3bb59511 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:12.489825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95368-17e3-49c5-a861-87ce3bb59511 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:12.489825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95368-17e3-49c5-a861-87ce3bb59511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:12.511946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b7a95368-17e3-49c5-a861-87ce3bb59511 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:45:12.513177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b7a95368-17e3-49c5-a861-87ce3bb59511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:45:12.618856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 622/1248] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:45:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:45:12.652840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f85310-ede4-4946-be31-65c325f934d7 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:12.807098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f85310-ede4-4946-be31-65c325f934d7 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:12.852920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504a883-78e9-4382-8cf5-fc1ef224bb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:12.853785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504a883-78e9-4382-8cf5-fc1ef224bb2b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:12.854067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504a883-78e9-4382-8cf5-fc1ef224bb2b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:12.854334 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504a883-78e9-4382-8cf5-fc1ef224bb2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:12.867017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504a883-78e9-4382-8cf5-fc1ef224bb2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-1504a883-78e9-4382-8cf5-fc1ef224bb2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:45:12.875528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:12.876888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:12.877819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 627/1249] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:45:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:12.883206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95368-17e3-49c5-a861-87ce3bb59511 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:13.038187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95368-17e3-49c5-a861-87ce3bb59511 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:13.097349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a020f83-d3bd-46db-80f3-ee59fe9dd88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:13.097960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020f83-d3bd-46db-80f3-ee59fe9dd88f None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:13.098353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020f83-d3bd-46db-80f3-ee59fe9dd88f None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:13.098514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020f83-d3bd-46db-80f3-ee59fe9dd88f None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:13.099545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a020f83-d3bd-46db-80f3-ee59fe9dd88f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:13.099854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a020f83-d3bd-46db-80f3-ee59fe9dd88f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:13.101704 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a020f83-d3bd-46db-80f3-ee59fe9dd88f None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:13.113366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 623/1250] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:45:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:45:13.120472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504a883-78e9-4382-8cf5-fc1ef224bb2b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:13.263307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504a883-78e9-4382-8cf5-fc1ef224bb2b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:13.299160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282c2e4-b347-40e2-9670-2ff3a0a8980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:13.299990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282c2e4-b347-40e2-9670-2ff3a0a8980c None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:13.300301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282c2e4-b347-40e2-9670-2ff3a0a8980c None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:13.300533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282c2e4-b347-40e2-9670-2ff3a0a8980c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:13.312658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282c2e4-b347-40e2-9670-2ff3a0a8980c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:13.314152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282c2e4-b347-40e2-9670-2ff3a0a8980c None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-8282c2e4-b347-40e2-9670-2ff3a0a8980c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:45:13.341718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:13.342895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:13.343981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:45:13.343981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:45:13.343981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:13.343981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:13.343981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 628/1251] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:13.350148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a020f83-d3bd-46db-80f3-ee59fe9dd88f None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:13.465080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a020f83-d3bd-46db-80f3-ee59fe9dd88f None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:13.508874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7414f3d-a356-47c3-a851-f81e871e9f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:13.509588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7414f3d-a356-47c3-a851-f81e871e9f22 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:13.509790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7414f3d-a356-47c3-a851-f81e871e9f22 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:13.510059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7414f3d-a356-47c3-a851-f81e871e9f22 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:13.510802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7414f3d-a356-47c3-a851-f81e871e9f22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:13.511105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7414f3d-a356-47c3-a851-f81e871e9f22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:13.512653 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7414f3d-a356-47c3-a851-f81e871e9f22 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:13.526476 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 624/1252] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:45:13.533246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282c2e4-b347-40e2-9670-2ff3a0a8980c None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:13.650892 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282c2e4-b347-40e2-9670-2ff3a0a8980c None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:13.688070 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ed11f-b880-4af8-926d-5058a8defd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:13.689163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ed11f-b880-4af8-926d-5058a8defd8b None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:13.689691 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ed11f-b880-4af8-926d-5058a8defd8b None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:13.690167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ed11f-b880-4af8-926d-5058a8defd8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:13.699815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ed11f-b880-4af8-926d-5058a8defd8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:13.701479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ed11f-b880-4af8-926d-5058a8defd8b None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-555ed11f-b880-4af8-926d-5058a8defd8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:45:13.718487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:13.719687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:13.720695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 629/1253] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:45:13.726164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7414f3d-a356-47c3-a851-f81e871e9f22 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:13.843928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7414f3d-a356-47c3-a851-f81e871e9f22 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:13.881179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d686-a6e6-48c3-abe7-4379b6810929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:13.881996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d686-a6e6-48c3-abe7-4379b6810929 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:13.881996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d686-a6e6-48c3-abe7-4379b6810929 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:13.882542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d686-a6e6-48c3-abe7-4379b6810929 None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:13.883974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d686-a6e6-48c3-abe7-4379b6810929 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:13.884286 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d686-a6e6-48c3-abe7-4379b6810929 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:13.886398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d686-a6e6-48c3-abe7-4379b6810929 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:13.914865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 625/1254] 10.222.0.79 () {56 vars in 1143 bytes} [Fri May 13 13:45:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:13.916037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ed11f-b880-4af8-926d-5058a8defd8b None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:14.041520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ed11f-b880-4af8-926d-5058a8defd8b None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:14.084980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d616d2-fd4f-4a72-a6c9-333c284d734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:14.085644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d616d2-fd4f-4a72-a6c9-333c284d734d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:14.085926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d616d2-fd4f-4a72-a6c9-333c284d734d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:14.086200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d616d2-fd4f-4a72-a6c9-333c284d734d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:14.177017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d616d2-fd4f-4a72-a6c9-333c284d734d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:14.178403 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d616d2-fd4f-4a72-a6c9-333c284d734d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:14.323567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 630/1255] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:45:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:45:14.331055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d686-a6e6-48c3-abe7-4379b6810929 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:14.490476 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d686-a6e6-48c3-abe7-4379b6810929 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:14.555081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32879b-b460-47b4-af1e-c10be7ff8ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:14.556938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32879b-b460-47b4-af1e-c10be7ff8ffb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:14.557413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32879b-b460-47b4-af1e-c10be7ff8ffb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:14.557916 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32879b-b460-47b4-af1e-c10be7ff8ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:14.577508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32879b-b460-47b4-af1e-c10be7ff8ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:14.580063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32879b-b460-47b4-af1e-c10be7ff8ffb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-bb32879b-b460-47b4-af1e-c10be7ff8ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:45:14.612040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:14.613328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:14.614507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:45:14.614507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:45:14.614507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:14.614507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:14.614507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 626/1256] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:14.621501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d616d2-fd4f-4a72-a6c9-333c284d734d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:14.761411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d616d2-fd4f-4a72-a6c9-333c284d734d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:14.803225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1442e9-e118-4a97-8548-e9d91a08dd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:14.803916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1442e9-e118-4a97-8548-e9d91a08dd93 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:14.804197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1442e9-e118-4a97-8548-e9d91a08dd93 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:14.804477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1442e9-e118-4a97-8548-e9d91a08dd93 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:14.805813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1442e9-e118-4a97-8548-e9d91a08dd93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:14.806137 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1442e9-e118-4a97-8548-e9d91a08dd93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:14.807575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1442e9-e118-4a97-8548-e9d91a08dd93 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:14.830773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 631/1257] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:14.838210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32879b-b460-47b4-af1e-c10be7ff8ffb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:14.972484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32879b-b460-47b4-af1e-c10be7ff8ffb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:15.028184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95be0c5-5b94-4dfc-84a8-b058131c7175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:15.028952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be0c5-5b94-4dfc-84a8-b058131c7175 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:15.030835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be0c5-5b94-4dfc-84a8-b058131c7175 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:15.030835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be0c5-5b94-4dfc-84a8-b058131c7175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:15.037543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95be0c5-5b94-4dfc-84a8-b058131c7175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:15.039128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95be0c5-5b94-4dfc-84a8-b058131c7175 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-d95be0c5-5b94-4dfc-84a8-b058131c7175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:45:15.056636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:15.058644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:15.060665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:45:15.060665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:45:15.060665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:45:15.060665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:45:15.060665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:45:15.060665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:45:15.060665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:45:15.060665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:15.060665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 627/1258] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:45:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:45:15.066696 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1442e9-e118-4a97-8548-e9d91a08dd93 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:15.227123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1442e9-e118-4a97-8548-e9d91a08dd93 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:15.284233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9af12-6692-48a1-b027-d891a3bf99b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:15.284835 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9af12-6692-48a1-b027-d891a3bf99b4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:15.285110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9af12-6692-48a1-b027-d891a3bf99b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:15.285384 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9af12-6692-48a1-b027-d891a3bf99b4 None admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:15.286167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9af12-6692-48a1-b027-d891a3bf99b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:15.286477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9af12-6692-48a1-b027-d891a3bf99b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:15.288232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9af12-6692-48a1-b027-d891a3bf99b4 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:15.304731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 632/1259] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:45:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:45:15.314359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95be0c5-5b94-4dfc-84a8-b058131c7175 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:15.473646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95be0c5-5b94-4dfc-84a8-b058131c7175 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:15.515865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f4670-a875-4230-b560-7f96457934f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:15.516536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f4670-a875-4230-b560-7f96457934f5 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:15.516814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f4670-a875-4230-b560-7f96457934f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:15.517084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f4670-a875-4230-b560-7f96457934f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:15.517990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f4670-a875-4230-b560-7f96457934f5 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=4ae9a6f1ac304e21b8d3115f2a0baf88) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:15.518301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f4670-a875-4230-b560-7f96457934f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:15.519904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f4670-a875-4230-b560-7f96457934f5 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:15.527853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 628/1260] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:45:15 2022] GET /identity/v3/role_assignments?user.id=4ae9a6f1ac304e21b8d3115f2a0baf88&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:17.908780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e11529-937b-4652-9fc4-b5318efa72f8 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:17.909346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e11529-937b-4652-9fc4-b5318efa72f8 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:17.909346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e11529-937b-4652-9fc4-b5318efa72f8 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:17.910735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 633/1261] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:45:17.917225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9c721-5ce5-499a-9cd8-7af9e62387ed None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:17.917751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9c721-5ce5-499a-9cd8-7af9e62387ed None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:17.917751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9c721-5ce5-499a-9cd8-7af9e62387ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:17.934181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c8c9c721-5ce5-499a-9cd8-7af9e62387ed None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:45:17.936058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c8c9c721-5ce5-499a-9cd8-7af9e62387ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:45:18.004264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 629/1262] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:45:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:45:18.027599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e11529-937b-4652-9fc4-b5318efa72f8 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:18.144142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e11529-937b-4652-9fc4-b5318efa72f8 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:18.184594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c41837-9492-4647-821b-3f2b9718d315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:18.185758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c41837-9492-4647-821b-3f2b9718d315 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:18.186158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c41837-9492-4647-821b-3f2b9718d315 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:18.186530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c41837-9492-4647-821b-3f2b9718d315 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:18.202842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c41837-9492-4647-821b-3f2b9718d315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-94c41837-9492-4647-821b-3f2b9718d315 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:45:18.211082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:18.212377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:18.213235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 634/1263] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:45:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:18.218375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9c721-5ce5-499a-9cd8-7af9e62387ed None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:18.331911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9c721-5ce5-499a-9cd8-7af9e62387ed None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:18.364519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bc26b-6983-4349-a3f3-3f6242841ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:18.365172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bc26b-6983-4349-a3f3-3f6242841ba7 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:18.365453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bc26b-6983-4349-a3f3-3f6242841ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:18.365733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bc26b-6983-4349-a3f3-3f6242841ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:18.366472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bc26b-6983-4349-a3f3-3f6242841ba7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:18.366803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bc26b-6983-4349-a3f3-3f6242841ba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:18.368760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bc26b-6983-4349-a3f3-3f6242841ba7 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:18.377074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 630/1264] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:45:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:18.382905 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c41837-9492-4647-821b-3f2b9718d315 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:18.478865 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c41837-9492-4647-821b-3f2b9718d315 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:18.510605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf621eec-83eb-4689-86dd-053b152fc4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:18.511411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf621eec-83eb-4689-86dd-053b152fc4d4 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:18.511685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf621eec-83eb-4689-86dd-053b152fc4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:18.511951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf621eec-83eb-4689-86dd-053b152fc4d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:18.522454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf621eec-83eb-4689-86dd-053b152fc4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:18.523949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf621eec-83eb-4689-86dd-053b152fc4d4 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-bf621eec-83eb-4689-86dd-053b152fc4d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:45:18.541065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:18.542249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:18.543247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:45:18.543247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:45:18.543247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:18.543247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:18.543247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 635/1265] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:18.548087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bc26b-6983-4349-a3f3-3f6242841ba7 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:18.657902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bc26b-6983-4349-a3f3-3f6242841ba7 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:18.723211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e00fb-5cd6-4709-93e1-1003b2fe3926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:18.724226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e00fb-5cd6-4709-93e1-1003b2fe3926 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:18.724647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e00fb-5cd6-4709-93e1-1003b2fe3926 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:18.725070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e00fb-5cd6-4709-93e1-1003b2fe3926 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:18.726300 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e00fb-5cd6-4709-93e1-1003b2fe3926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:18.727387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e00fb-5cd6-4709-93e1-1003b2fe3926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:18.729694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e00fb-5cd6-4709-93e1-1003b2fe3926 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:18.751010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 631/1266] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:18.760063 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf621eec-83eb-4689-86dd-053b152fc4d4 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:18.874620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf621eec-83eb-4689-86dd-053b152fc4d4 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:18.907094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bd1ac-d640-49fa-b74b-95004b0164fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:18.907892 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bd1ac-d640-49fa-b74b-95004b0164fa None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:18.908169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bd1ac-d640-49fa-b74b-95004b0164fa None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:18.908441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bd1ac-d640-49fa-b74b-95004b0164fa None admin] PATH_INFO: `/v3/users/placement` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:18.916763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604bd1ac-d640-49fa-b74b-95004b0164fa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:18.918203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604bd1ac-d640-49fa-b74b-95004b0164fa None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-604bd1ac-d640-49fa-b74b-95004b0164fa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:45:18.932499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:18.933627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:18.934859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 636/1267] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:18.939622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e00fb-5cd6-4709-93e1-1003b2fe3926 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:19.037667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e00fb-5cd6-4709-93e1-1003b2fe3926 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:19.070261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9b26d-1a6e-43ee-b31a-0ed43edd67bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:19.070960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9b26d-1a6e-43ee-b31a-0ed43edd67bc None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:19.071240 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9b26d-1a6e-43ee-b31a-0ed43edd67bc None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:19.071508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9b26d-1a6e-43ee-b31a-0ed43edd67bc None admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:19.072322 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9b26d-1a6e-43ee-b31a-0ed43edd67bc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:19.072617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9b26d-1a6e-43ee-b31a-0ed43edd67bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:19.074022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9b26d-1a6e-43ee-b31a-0ed43edd67bc None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:19.090639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 632/1268] 10.222.0.79 () {56 vars in 1143 bytes} [Fri May 13 13:45:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:19.096445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bd1ac-d640-49fa-b74b-95004b0164fa None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:19.209485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bd1ac-d640-49fa-b74b-95004b0164fa None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:19.246159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510dc3b-e7c0-4ad6-88e4-e89acb8c38f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:19.246982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510dc3b-e7c0-4ad6-88e4-e89acb8c38f2 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:19.247258 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510dc3b-e7c0-4ad6-88e4-e89acb8c38f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:19.247526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510dc3b-e7c0-4ad6-88e4-e89acb8c38f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:19.259221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510dc3b-e7c0-4ad6-88e4-e89acb8c38f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:19.260588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510dc3b-e7c0-4ad6-88e4-e89acb8c38f2 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-3510dc3b-e7c0-4ad6-88e4-e89acb8c38f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:45:19.279871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:19.281701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:19.283473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:45:19.283473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:45:19.283473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:19.283473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:19.283473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 637/1269] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:19.287079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9b26d-1a6e-43ee-b31a-0ed43edd67bc None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:19.404801 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9b26d-1a6e-43ee-b31a-0ed43edd67bc None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:19.449092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5690642-f22e-46e0-952c-8cadca8d6946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:19.449751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5690642-f22e-46e0-952c-8cadca8d6946 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:19.450028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5690642-f22e-46e0-952c-8cadca8d6946 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:19.450299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5690642-f22e-46e0-952c-8cadca8d6946 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:19.451049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5690642-f22e-46e0-952c-8cadca8d6946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:19.451354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5690642-f22e-46e0-952c-8cadca8d6946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:19.452724 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5690642-f22e-46e0-952c-8cadca8d6946 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:19.466232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 633/1270] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:19.472202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510dc3b-e7c0-4ad6-88e4-e89acb8c38f2 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:19.574284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510dc3b-e7c0-4ad6-88e4-e89acb8c38f2 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:19.608784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434a004-3733-4fa0-92a6-35ef925c9e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:19.609567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434a004-3733-4fa0-92a6-35ef925c9e2d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:19.609843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434a004-3733-4fa0-92a6-35ef925c9e2d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:19.610113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434a004-3733-4fa0-92a6-35ef925c9e2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:19.616914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434a004-3733-4fa0-92a6-35ef925c9e2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:19.618928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434a004-3733-4fa0-92a6-35ef925c9e2d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-e434a004-3733-4fa0-92a6-35ef925c9e2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:45:19.633675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:19.634841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:19.635896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:45:19.635896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:45:19.635896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:45:19.635896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:45:19.635896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:45:19.635896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:45:19.635896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:45:19.635896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:19.635896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 638/1271] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:45:19.641306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5690642-f22e-46e0-952c-8cadca8d6946 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:19.747418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5690642-f22e-46e0-952c-8cadca8d6946 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:19.781042 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b0794-22ac-483e-8d1b-3366f844b70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:19.781690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b0794-22ac-483e-8d1b-3366f844b70a None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:19.781972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b0794-22ac-483e-8d1b-3366f844b70a None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:19.782249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b0794-22ac-483e-8d1b-3366f844b70a None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:19.783058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b0794-22ac-483e-8d1b-3366f844b70a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:19.783359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b0794-22ac-483e-8d1b-3366f844b70a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:19.784798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b0794-22ac-483e-8d1b-3366f844b70a None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:19.797695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 634/1272] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:19.803980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434a004-3733-4fa0-92a6-35ef925c9e2d None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:19.916816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434a004-3733-4fa0-92a6-35ef925c9e2d None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:19.956033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdcff6-f40f-40a3-addd-383f4184cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:19.956931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdcff6-f40f-40a3-addd-383f4184cbb8 None admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:19.957207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdcff6-f40f-40a3-addd-383f4184cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:19.957479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdcff6-f40f-40a3-addd-383f4184cbb8 None admin] PATH_INFO: `/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/4ae9a6f1ac304e21b8d3115f2a0baf88/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:19.959810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdcff6-f40f-40a3-addd-383f4184cbb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=129cd7365a3f45a1ba4c0308a6b1cacd, user_id=4ae9a6f1ac304e21b8d3115f2a0baf88, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:19.961843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdcff6-f40f-40a3-addd-383f4184cbb8 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:19.979607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 639/1273] 10.222.0.79 () {58 vars in 1439 bytes} [Fri May 13 13:45:19 2022] PUT /identity/v3/projects/129cd7365a3f45a1ba4c0308a6b1cacd/users/4ae9a6f1ac304e21b8d3115f2a0baf88/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:45:22.415470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d94f83-5c4e-426c-8a0a-3028ba87a49c None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:22.416078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d94f83-5c4e-426c-8a0a-3028ba87a49c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:22.416078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d94f83-5c4e-426c-8a0a-3028ba87a49c None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:22.417439 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 635/1274] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:45:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:45:22.424074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22daf01-43f8-440b-92b4-b2cf5d3a9350 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:22.424790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22daf01-43f8-440b-92b4-b2cf5d3a9350 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:22.424790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22daf01-43f8-440b-92b4-b2cf5d3a9350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:22.443343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e22daf01-43f8-440b-92b4-b2cf5d3a9350 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:45:22.444641 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e22daf01-43f8-440b-92b4-b2cf5d3a9350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:45:22.527289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 640/1275] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:45:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:45:22.553417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d94f83-5c4e-426c-8a0a-3028ba87a49c None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:22.654873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d94f83-5c4e-426c-8a0a-3028ba87a49c None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:22.687936 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526fef0-8cda-4ca8-87d3-960e6c4da640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:22.688722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526fef0-8cda-4ca8-87d3-960e6c4da640 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:22.688996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526fef0-8cda-4ca8-87d3-960e6c4da640 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:22.689266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526fef0-8cda-4ca8-87d3-960e6c4da640 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:22.700275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526fef0-8cda-4ca8-87d3-960e6c4da640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-9526fef0-8cda-4ca8-87d3-960e6c4da640 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:45:22.707951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:22.709086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:22.709978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 636/1276] 10.222.0.79 () {56 vars in 1096 bytes} [Fri May 13 13:45:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:22.716308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22daf01-43f8-440b-92b4-b2cf5d3a9350 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:22.813537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22daf01-43f8-440b-92b4-b2cf5d3a9350 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:22.847067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb229238-8ab5-4a0f-9f92-9c6d740a3c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:22.847722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229238-8ab5-4a0f-9f92-9c6d740a3c37 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:22.848002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229238-8ab5-4a0f-9f92-9c6d740a3c37 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:22.848273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229238-8ab5-4a0f-9f92-9c6d740a3c37 None admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:22.849001 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb229238-8ab5-4a0f-9f92-9c6d740a3c37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:22.849307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb229238-8ab5-4a0f-9f92-9c6d740a3c37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:22.850663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb229238-8ab5-4a0f-9f92-9c6d740a3c37 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:22.858955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 641/1277] 10.222.0.79 () {56 vars in 1099 bytes} [Fri May 13 13:45:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:22.864768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526fef0-8cda-4ca8-87d3-960e6c4da640 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:22.962002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526fef0-8cda-4ca8-87d3-960e6c4da640 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:22.993878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef968f-b6dd-4932-8b19-806dddd61538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:22.994666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef968f-b6dd-4932-8b19-806dddd61538 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:22.994982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef968f-b6dd-4932-8b19-806dddd61538 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:22.995259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef968f-b6dd-4932-8b19-806dddd61538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:23.005280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef968f-b6dd-4932-8b19-806dddd61538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:23.006774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef968f-b6dd-4932-8b19-806dddd61538 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-94ef968f-b6dd-4932-8b19-806dddd61538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:45:23.024145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:23.025247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:23.026362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:45:23.026362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:45:23.026362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:23.026362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:23.026362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 637/1278] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:23.031178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb229238-8ab5-4a0f-9f92-9c6d740a3c37 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:23.137302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb229238-8ab5-4a0f-9f92-9c6d740a3c37 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:23.174054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005db23-f1b3-43d6-ab7f-80e28f16c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:23.174744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005db23-f1b3-43d6-ab7f-80e28f16c740 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:23.175034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005db23-f1b3-43d6-ab7f-80e28f16c740 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:23.175417 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005db23-f1b3-43d6-ab7f-80e28f16c740 None admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:23.176222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005db23-f1b3-43d6-ab7f-80e28f16c740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:23.176558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005db23-f1b3-43d6-ab7f-80e28f16c740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:23.177940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005db23-f1b3-43d6-ab7f-80e28f16c740 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:23.194310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 642/1279] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:23.201135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef968f-b6dd-4932-8b19-806dddd61538 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:23.317308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef968f-b6dd-4932-8b19-806dddd61538 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:23.351362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38508098-1476-40eb-b473-1b0afd95ea48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:23.352305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38508098-1476-40eb-b473-1b0afd95ea48 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:23.352579 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38508098-1476-40eb-b473-1b0afd95ea48 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:23.352846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38508098-1476-40eb-b473-1b0afd95ea48 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:23.360656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38508098-1476-40eb-b473-1b0afd95ea48 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:23.362108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38508098-1476-40eb-b473-1b0afd95ea48 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-38508098-1476-40eb-b473-1b0afd95ea48 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:45:23.377100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:23.379663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:23.381902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 638/1280] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:23.385657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005db23-f1b3-43d6-ab7f-80e28f16c740 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:23.505977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005db23-f1b3-43d6-ab7f-80e28f16c740 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:23.541698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3538e81-e968-42c9-8579-ad3e2785905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:23.542363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3538e81-e968-42c9-8579-ad3e2785905a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:23.542644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3538e81-e968-42c9-8579-ad3e2785905a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:23.542939 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3538e81-e968-42c9-8579-ad3e2785905a None admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:23.543766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3538e81-e968-42c9-8579-ad3e2785905a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:23.544247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3538e81-e968-42c9-8579-ad3e2785905a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:23.545692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3538e81-e968-42c9-8579-ad3e2785905a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:23.565471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 643/1281] 10.222.0.79 () {56 vars in 1143 bytes} [Fri May 13 13:45:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:23.572266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38508098-1476-40eb-b473-1b0afd95ea48 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:23.680603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38508098-1476-40eb-b473-1b0afd95ea48 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:23.713861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86c6f6-e1a6-4520-8eb8-7ba822158762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:23.714507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86c6f6-e1a6-4520-8eb8-7ba822158762 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:23.714816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86c6f6-e1a6-4520-8eb8-7ba822158762 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:23.715095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86c6f6-e1a6-4520-8eb8-7ba822158762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:23.779958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86c6f6-e1a6-4520-8eb8-7ba822158762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:23.781397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86c6f6-e1a6-4520-8eb8-7ba822158762 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:23.897150 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 639/1282] 10.222.0.79 () {58 vars in 1282 bytes} [Fri May 13 13:45:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:45:23.903869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3538e81-e968-42c9-8579-ad3e2785905a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:24.019411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3538e81-e968-42c9-8579-ad3e2785905a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:24.055508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865ef7d-fc5b-4d92-bd26-e9bbea133233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:24.056313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865ef7d-fc5b-4d92-bd26-e9bbea133233 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:24.056602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865ef7d-fc5b-4d92-bd26-e9bbea133233 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:24.056876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865ef7d-fc5b-4d92-bd26-e9bbea133233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:24.067135 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865ef7d-fc5b-4d92-bd26-e9bbea133233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:24.068574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865ef7d-fc5b-4d92-bd26-e9bbea133233 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-7865ef7d-fc5b-4d92-bd26-e9bbea133233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:45:24.087139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:24.088283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:24.089481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:45:24.089481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:45:24.089481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:45:24.089481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:24.089481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 644/1283] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:45:24.094094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86c6f6-e1a6-4520-8eb8-7ba822158762 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:24.206666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86c6f6-e1a6-4520-8eb8-7ba822158762 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:24.240389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae440e-e7b2-484a-9766-44b0f1a38a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:24.241041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae440e-e7b2-484a-9766-44b0f1a38a60 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:24.241317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae440e-e7b2-484a-9766-44b0f1a38a60 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:24.241585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae440e-e7b2-484a-9766-44b0f1a38a60 None admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:24.242881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae440e-e7b2-484a-9766-44b0f1a38a60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:24.243161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae440e-e7b2-484a-9766-44b0f1a38a60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:24.244535 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae440e-e7b2-484a-9766-44b0f1a38a60 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:24.256274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 640/1284] 10.222.0.79 () {56 vars in 1109 bytes} [Fri May 13 13:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:24.262954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865ef7d-fc5b-4d92-bd26-e9bbea133233 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:24.364579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865ef7d-fc5b-4d92-bd26-e9bbea133233 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:24.398767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b32ac1-5c97-4a1a-b0d4-dd35be9ade88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:24.399554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b32ac1-5c97-4a1a-b0d4-dd35be9ade88 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:24.399830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b32ac1-5c97-4a1a-b0d4-dd35be9ade88 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:24.400098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b32ac1-5c97-4a1a-b0d4-dd35be9ade88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:24.406593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b32ac1-5c97-4a1a-b0d4-dd35be9ade88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:24.408057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b32ac1-5c97-4a1a-b0d4-dd35be9ade88 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application [None req-f2b32ac1-5c97-4a1a-b0d4-dd35be9ade88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:45:24.421823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application with Lock( May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:24.422927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:24.424146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:45:24.424146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:45:24.424146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:45:24.424146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:45:24.424146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:45:24.424146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:45:24.424146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:45:24.424146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: ERROR keystone.server.flask.application May 13 13:45:24.424146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 645/1285] 10.222.0.79 () {56 vars in 1111 bytes} [Fri May 13 13:45:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:24.428918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae440e-e7b2-484a-9766-44b0f1a38a60 None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:24.527473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae440e-e7b2-484a-9766-44b0f1a38a60 None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:24.560016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc2be9-7f46-429d-82b7-14490085ec19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:24.560686 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc2be9-7f46-429d-82b7-14490085ec19 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:24.560969 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc2be9-7f46-429d-82b7-14490085ec19 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:24.561237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc2be9-7f46-429d-82b7-14490085ec19 None admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:24.562025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc2be9-7f46-429d-82b7-14490085ec19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:24.562331 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc2be9-7f46-429d-82b7-14490085ec19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:24.563806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc2be9-7f46-429d-82b7-14490085ec19 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:24.579414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 641/1286] 10.222.0.79 () {56 vars in 1148 bytes} [Fri May 13 13:45:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:24.586829 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b32ac1-5c97-4a1a-b0d4-dd35be9ade88 None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:24.687528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b32ac1-5c97-4a1a-b0d4-dd35be9ade88 None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:24.739006 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac285a-8513-4208-a028-55cbded14444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:24.740026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac285a-8513-4208-a028-55cbded14444 None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:24.740453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac285a-8513-4208-a028-55cbded14444 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:24.740898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac285a-8513-4208-a028-55cbded14444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:24.742965 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac285a-8513-4208-a028-55cbded14444 None admin] RBAC: Adding query filter params (role.id=4416a11ddf5d4ed194c121f0e9ebf9b3, scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd, user.id=4ae9a6f1ac304e21b8d3115f2a0baf88) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:24.743499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac285a-8513-4208-a028-55cbded14444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:24.745809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac285a-8513-4208-a028-55cbded14444 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:24.755398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 646/1287] 10.222.0.79 () {56 vars in 1374 bytes} [Fri May 13 13:45:24 2022] GET /identity/v3/role_assignments?user.id=4ae9a6f1ac304e21b8d3115f2a0baf88&scope.project.id=129cd7365a3f45a1ba4c0308a6b1cacd&role.id=4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:45:27.072310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7edf4-8818-4179-810d-79a3c43a4087 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:27.072860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7edf4-8818-4179-810d-79a3c43a4087 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:27.072860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7edf4-8818-4179-810d-79a3c43a4087 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:27.074192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 642/1288] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:45:27.079623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3409470-fb5e-45f2-be85-45dc03d03a99 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:27.080068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3409470-fb5e-45f2-be85-45dc03d03a99 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:27.080163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3409470-fb5e-45f2-be85-45dc03d03a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:27.097966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b3409470-fb5e-45f2-be85-45dc03d03a99 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:45:27.099197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b3409470-fb5e-45f2-be85-45dc03d03a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:45:27.166869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 647/1289] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:45:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:45:27.194065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7edf4-8818-4179-810d-79a3c43a4087 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:27.312185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7edf4-8818-4179-810d-79a3c43a4087 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:27.354410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44d7c-4b73-4c9a-8011-a6356d1ba9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:27.355023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44d7c-4b73-4c9a-8011-a6356d1ba9af None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:27.355299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44d7c-4b73-4c9a-8011-a6356d1ba9af None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:27.355565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44d7c-4b73-4c9a-8011-a6356d1ba9af None admin] PATH_INFO: `/v3/services/placement` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:27.356206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44d7c-4b73-4c9a-8011-a6356d1ba9af None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:27.357564 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44d7c-4b73-4c9a-8011-a6356d1ba9af None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-3dc44d7c-4b73-4c9a-8011-a6356d1ba9af None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:27.369101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:27.370312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:27.371386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 643/1290] 10.222.0.79 () {56 vars in 1117 bytes} [Fri May 13 13:45:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:27.376512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3409470-fb5e-45f2-be85-45dc03d03a99 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:27.483256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3409470-fb5e-45f2-be85-45dc03d03a99 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:27.519274 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9dec8-613b-4a36-a169-e1e52f0c848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:27.519893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9dec8-613b-4a36-a169-e1e52f0c848d None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:27.520169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9dec8-613b-4a36-a169-e1e52f0c848d None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:27.520438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9dec8-613b-4a36-a169-e1e52f0c848d None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:27.521136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9dec8-613b-4a36-a169-e1e52f0c848d None admin] RBAC: Adding query filter params (name=placement) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:27.521433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9dec8-613b-4a36-a169-e1e52f0c848d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:27.522821 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9dec8-613b-4a36-a169-e1e52f0c848d None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:27.528856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 648/1291] 10.222.0.79 () {56 vars in 1116 bytes} [Fri May 13 13:45:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:27.534694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44d7c-4b73-4c9a-8011-a6356d1ba9af None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:27.655689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44d7c-4b73-4c9a-8011-a6356d1ba9af None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:27.693429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7c991-65f8-4a86-bc53-d3fe74862de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:27.694082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7c991-65f8-4a86-bc53-d3fe74862de8 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:27.694361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7c991-65f8-4a86-bc53-d3fe74862de8 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:27.694634 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7c991-65f8-4a86-bc53-d3fe74862de8 None admin] PATH_INFO: `/v3/services` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:27.695892 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7c991-65f8-4a86-bc53-d3fe74862de8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:27.696196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7c991-65f8-4a86-bc53-d3fe74862de8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:27.697611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7c991-65f8-4a86-bc53-d3fe74862de8 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:27.702015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 644/1292] 10.222.0.79 () {56 vars in 1116 bytes} [Fri May 13 13:45:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:30.010802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd06a8e-3156-448f-bed1-51bb78031fd3 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:30.011350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd06a8e-3156-448f-bed1-51bb78031fd3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:30.011350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd06a8e-3156-448f-bed1-51bb78031fd3 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:30.012675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 649/1293] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:45:30.017998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf509d-112d-4ee0-9043-dd245b8d33a8 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:30.018475 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf509d-112d-4ee0-9043-dd245b8d33a8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:30.018581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf509d-112d-4ee0-9043-dd245b8d33a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:30.035439 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-6fcf509d-112d-4ee0-9043-dd245b8d33a8 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:45:30.036642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6fcf509d-112d-4ee0-9043-dd245b8d33a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:45:30.105888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 645/1294] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:45:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:45:30.143491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd06a8e-3156-448f-bed1-51bb78031fd3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:30.259337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd06a8e-3156-448f-bed1-51bb78031fd3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:30.292889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea441b-5283-43a8-a476-d0c938630c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:30.293552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea441b-5283-43a8-a476-d0c938630c25 None admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:30.293831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea441b-5283-43a8-a476-d0c938630c25 None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:30.294102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea441b-5283-43a8-a476-d0c938630c25 None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:30.294843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea441b-5283-43a8-a476-d0c938630c25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:30.296212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea441b-5283-43a8-a476-d0c938630c25 None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:30.309999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 650/1295] 10.222.0.79 () {60 vars in 1141 bytes} [Fri May 13 13:45:30 2022] POST /identity/v3/services => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:45:32.558899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31ec68-d6f7-4e01-9fa3-5ca199a9707e None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:32.559488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31ec68-d6f7-4e01-9fa3-5ca199a9707e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:32.559488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31ec68-d6f7-4e01-9fa3-5ca199a9707e None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:32.560828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 646/1296] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:45:32.566600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e134c-2b55-44fe-b8e0-faf49c4edc67 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:32.567182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e134c-2b55-44fe-b8e0-faf49c4edc67 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:32.567182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e134c-2b55-44fe-b8e0-faf49c4edc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:32.584655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-dd1e134c-2b55-44fe-b8e0-faf49c4edc67 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:45:32.585872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-dd1e134c-2b55-44fe-b8e0-faf49c4edc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:45:32.658653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 651/1297] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:45:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:45:32.682090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31ec68-d6f7-4e01-9fa3-5ca199a9707e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:32.780409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31ec68-d6f7-4e01-9fa3-5ca199a9707e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:32.818762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ddff4-2313-47f5-b70b-e79174eac98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:32.819430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ddff4-2313-47f5-b70b-e79174eac98c None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:32.819728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ddff4-2313-47f5-b70b-e79174eac98c None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:32.820013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ddff4-2313-47f5-b70b-e79174eac98c None admin] PATH_INFO: `/v3/services/placement` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:32.820653 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ddff4-2313-47f5-b70b-e79174eac98c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:32.822043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ddff4-2313-47f5-b70b-e79174eac98c None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-278ddff4-2313-47f5-b70b-e79174eac98c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:32.833668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:32.834923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:32.835953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 647/1298] 10.222.0.79 () {56 vars in 1117 bytes} [Fri May 13 13:45:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:32.840986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e134c-2b55-44fe-b8e0-faf49c4edc67 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:32.933488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e134c-2b55-44fe-b8e0-faf49c4edc67 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:32.965642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9288ad5-4ab8-4af2-a744-19285b3ca79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:32.966301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9288ad5-4ab8-4af2-a744-19285b3ca79a None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:32.966582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9288ad5-4ab8-4af2-a744-19285b3ca79a None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:32.966889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9288ad5-4ab8-4af2-a744-19285b3ca79a None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:32.967587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9288ad5-4ab8-4af2-a744-19285b3ca79a None admin] RBAC: Adding query filter params (name=placement) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:32.967891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9288ad5-4ab8-4af2-a744-19285b3ca79a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:32.969262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9288ad5-4ab8-4af2-a744-19285b3ca79a None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:32.974404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 652/1299] 10.222.0.79 () {56 vars in 1116 bytes} [Fri May 13 13:45:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:32.981843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ddff4-2313-47f5-b70b-e79174eac98c None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:33.089218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ddff4-2313-47f5-b70b-e79174eac98c None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:33.123863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f0cec-5b23-4dac-a86f-a29ec9edd6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:33.124530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f0cec-5b23-4dac-a86f-a29ec9edd6a1 None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:33.124811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f0cec-5b23-4dac-a86f-a29ec9edd6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:33.125085 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f0cec-5b23-4dac-a86f-a29ec9edd6a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:33.125871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f0cec-5b23-4dac-a86f-a29ec9edd6a1 None admin] RBAC: Adding query filter params (interface=public, service_id=08153d1091aa47dc80f3510a6462fab2, region_id=RegionOne) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:33.126182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f0cec-5b23-4dac-a86f-a29ec9edd6a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:33.127598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f0cec-5b23-4dac-a86f-a29ec9edd6a1 None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:33.132883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 648/1300] 10.222.0.79 () {56 vars in 1251 bytes} [Fri May 13 13:45:32 2022] GET /identity/v3/endpoints?service_id=08153d1091aa47dc80f3510a6462fab2&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:33.138835 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9288ad5-4ab8-4af2-a744-19285b3ca79a None admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:33.247204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9288ad5-4ab8-4af2-a744-19285b3ca79a None admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:33.280911 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366bfa3-6f12-432b-b1ac-fd3ec653baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:33.281569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366bfa3-6f12-432b-b1ac-fd3ec653baac None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:33.281850 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366bfa3-6f12-432b-b1ac-fd3ec653baac None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:33.282125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366bfa3-6f12-432b-b1ac-fd3ec653baac None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:33.282831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366bfa3-6f12-432b-b1ac-fd3ec653baac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:33.284208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366bfa3-6f12-432b-b1ac-fd3ec653baac None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:33.290960 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 653/1301] 10.222.0.79 () {56 vars in 1087 bytes} [Fri May 13 13:45:33 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:45:35.557864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555eb14-a614-4620-b5c5-2ec6aac8a229 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:35.558432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555eb14-a614-4620-b5c5-2ec6aac8a229 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:35.558432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555eb14-a614-4620-b5c5-2ec6aac8a229 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:35.559849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 649/1302] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:45:35.566773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978f12-85a4-4b0a-a4a5-ca3daa3ef963 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:35.567289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978f12-85a4-4b0a-a4a5-ca3daa3ef963 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:35.567661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978f12-85a4-4b0a-a4a5-ca3daa3ef963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:35.589249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-d1978f12-85a4-4b0a-a4a5-ca3daa3ef963 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:45:35.590483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d1978f12-85a4-4b0a-a4a5-ca3daa3ef963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:45:35.661573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 654/1303] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:45:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:45:35.689392 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555eb14-a614-4620-b5c5-2ec6aac8a229 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:35.808702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555eb14-a614-4620-b5c5-2ec6aac8a229 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:35.846008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24b620-7bd0-4137-b25e-8fb3ad3d3fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:35.847422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24b620-7bd0-4137-b25e-8fb3ad3d3fbb None admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:35.847913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24b620-7bd0-4137-b25e-8fb3ad3d3fbb None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:35.848880 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24b620-7bd0-4137-b25e-8fb3ad3d3fbb None admin] PATH_INFO: `/v3/services/placement` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:35.849611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24b620-7bd0-4137-b25e-8fb3ad3d3fbb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:35.852260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24b620-7bd0-4137-b25e-8fb3ad3d3fbb None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application [None req-ad24b620-7bd0-4137-b25e-8fb3ad3d3fbb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:45:35.874955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:45:35.876102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application with Lock( May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self._enter() May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application return self.creator() May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application created_value = creator( May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:45:35.876960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: ERROR keystone.server.flask.application May 13 13:45:35.877802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 650/1304] 10.222.0.79 () {56 vars in 1117 bytes} [Fri May 13 13:45:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:45:35.885183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978f12-85a4-4b0a-a4a5-ca3daa3ef963 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:36.010069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978f12-85a4-4b0a-a4a5-ca3daa3ef963 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:36.044513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9eb4a0-9783-45ec-81ce-3f97352fa70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:36.045174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eb4a0-9783-45ec-81ce-3f97352fa70e None admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:36.045455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eb4a0-9783-45ec-81ce-3f97352fa70e None admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:36.045724 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eb4a0-9783-45ec-81ce-3f97352fa70e None admin] PATH_INFO: `/v3/services` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:36.046424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9eb4a0-9783-45ec-81ce-3f97352fa70e None admin] RBAC: Adding query filter params (name=placement) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:45:36.046750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9eb4a0-9783-45ec-81ce-3f97352fa70e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:36.048114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9eb4a0-9783-45ec-81ce-3f97352fa70e None admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:36.053555 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 655/1305] 10.222.0.79 () {56 vars in 1116 bytes} [Fri May 13 13:45:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:45:36.068413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24b620-7bd0-4137-b25e-8fb3ad3d3fbb None admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:45:36.177687 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24b620-7bd0-4137-b25e-8fb3ad3d3fbb None admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:45:36.214099 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b386ba-a0d3-43d2-848b-670ca240a9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:45:36.214804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b386ba-a0d3-43d2-848b-670ca240a9ec None admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:45:36.215086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b386ba-a0d3-43d2-848b-670ca240a9ec None admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:45:36.215359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b386ba-a0d3-43d2-848b-670ca240a9ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:45:36.216094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b386ba-a0d3-43d2-848b-670ca240a9ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:45:36.217458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b386ba-a0d3-43d2-848b-670ca240a9ec None admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:45:36.236011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 651/1306] 10.222.0.79 () {60 vars in 1144 bytes} [Fri May 13 13:45:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:46:45.819966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b525f7-7055-4dfc-9df4-dc0d37a8d918 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:45.821046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b525f7-7055-4dfc-9df4-dc0d37a8d918 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:45.821530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b525f7-7055-4dfc-9df4-dc0d37a8d918 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:45.827784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 656/1307] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:46:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:46:45.829301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501bcd8-88e1-4d54-b7e8-651a615dcf7b None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:45.829785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501bcd8-88e1-4d54-b7e8-651a615dcf7b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:45.829869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501bcd8-88e1-4d54-b7e8-651a615dcf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:45.919079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-4501bcd8-88e1-4d54-b7e8-651a615dcf7b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:46:45.920319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4501bcd8-88e1-4d54-b7e8-651a615dcf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:46.055807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 652/1308] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:46:46.108414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a88e-7e27-4eef-b4c7-060d5e9bb5a2 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:46.108564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a88e-7e27-4eef-b4c7-060d5e9bb5a2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:46.109286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a88e-7e27-4eef-b4c7-060d5e9bb5a2 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:46.112693 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 657/1309] 10.222.0.79 () {54 vars in 964 bytes} [Fri May 13 13:46:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:46:46.121365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02c679-085e-4eaa-8d8f-b8e06f38b6a6 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:46.121853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02c679-085e-4eaa-8d8f-b8e06f38b6a6 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:46.122134 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02c679-085e-4eaa-8d8f-b8e06f38b6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:46.268013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-0c02c679-085e-4eaa-8d8f-b8e06f38b6a6 None None] MFA Rules not processed for user `8f784cc461ab48ad92140680758a3704`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:46:46.269739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0c02c679-085e-4eaa-8d8f-b8e06f38b6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:46.378667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 653/1310] 10.222.0.79 () {58 vars in 1061 bytes} [Fri May 13 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:46:46.389838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a88e-7e27-4eef-b4c7-060d5e9bb5a2 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:46:46.392682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5620a88e-7e27-4eef-b4c7-060d5e9bb5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:46.525363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a88e-7e27-4eef-b4c7-060d5e9bb5a2 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:46:46.611838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d36265-3fbc-4b59-873b-ed176052b7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:46:46.612598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d36265-3fbc-4b59-873b-ed176052b7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:46.612872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d36265-3fbc-4b59-873b-ed176052b7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:46.613168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d36265-3fbc-4b59-873b-ed176052b7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:46.615019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-43d36265-3fbc-4b59-873b-ed176052b7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:46.733218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d36265-3fbc-4b59-873b-ed176052b7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:46:46.735168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d36265-3fbc-4b59-873b-ed176052b7f5 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:46:46.765446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 658/1311] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:46:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:46:49.630301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288839-036a-4363-b0ea-4461bccd9c5e None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:49.630858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288839-036a-4363-b0ea-4461bccd9c5e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:49.630999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288839-036a-4363-b0ea-4461bccd9c5e None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:49.633543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 654/1312] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:46:49.639298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8005f03-86fd-4f27-bfb6-01cd87eaa63f None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:49.639823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8005f03-86fd-4f27-bfb6-01cd87eaa63f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:49.639823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8005f03-86fd-4f27-bfb6-01cd87eaa63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:49.742272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a8005f03-86fd-4f27-bfb6-01cd87eaa63f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:46:49.744126 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a8005f03-86fd-4f27-bfb6-01cd87eaa63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:49.867292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 659/1313] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:46:49.906846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89288839-036a-4363-b0ea-4461bccd9c5e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:46:49.908546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-89288839-036a-4363-b0ea-4461bccd9c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:50.055883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89288839-036a-4363-b0ea-4461bccd9c5e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:46:50.143283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1e684-c0db-4e4c-bdf0-8a6da51c0c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:46:50.143993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1e684-c0db-4e4c-bdf0-8a6da51c0c8b service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:50.144274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1e684-c0db-4e4c-bdf0-8a6da51c0c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:50.144546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1e684-c0db-4e4c-bdf0-8a6da51c0c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:50.146267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-43f1e684-c0db-4e4c-bdf0-8a6da51c0c8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:50.282162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1e684-c0db-4e4c-bdf0-8a6da51c0c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:46:50.283665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1e684-c0db-4e4c-bdf0-8a6da51c0c8b service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:46:50.306907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 655/1314] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:46:53.056922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4f837-c362-4210-aa05-7e0d0199663b None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:53.057468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4f837-c362-4210-aa05-7e0d0199663b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:53.057468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4f837-c362-4210-aa05-7e0d0199663b None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:53.058858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 660/1315] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:46:53.064888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd4011-dbc5-452f-acde-fefdfc76ed99 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:53.065797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd4011-dbc5-452f-acde-fefdfc76ed99 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:53.065797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd4011-dbc5-452f-acde-fefdfc76ed99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:53.137660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-2ddd4011-dbc5-452f-acde-fefdfc76ed99 None None] MFA Rules not processed for user `adb0be7f832d47cfa9b23b4532305e94`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:46:53.138956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2ddd4011-dbc5-452f-acde-fefdfc76ed99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:53.248108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 656/1316] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:46:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:46:53.289953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4f837-c362-4210-aa05-7e0d0199663b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:46:53.291698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f2e4f837-c362-4210-aa05-7e0d0199663b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:53.407374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4f837-c362-4210-aa05-7e0d0199663b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:46:53.500362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e391372-b331-4bd4-9275-008298e2a434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:46:53.500922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e391372-b331-4bd4-9275-008298e2a434 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:53.501202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e391372-b331-4bd4-9275-008298e2a434 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:53.501480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e391372-b331-4bd4-9275-008298e2a434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:53.503226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-4e391372-b331-4bd4-9275-008298e2a434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:53.637644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e391372-b331-4bd4-9275-008298e2a434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:46:53.639148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e391372-b331-4bd4-9275-008298e2a434 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:46:53.659553 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 661/1317] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:46:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:46:56.736720 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f51f5c-d342-4f87-a8a8-eeda7f6e2107 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:56.737312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f51f5c-d342-4f87-a8a8-eeda7f6e2107 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:56.737312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f51f5c-d342-4f87-a8a8-eeda7f6e2107 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:56.738662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 657/1318] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:46:56.745655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eafa2f-52d0-415c-8593-82e18693f7df None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:56.746184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eafa2f-52d0-415c-8593-82e18693f7df None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:56.746272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eafa2f-52d0-415c-8593-82e18693f7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:56.827461 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a1eafa2f-52d0-415c-8593-82e18693f7df None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:46:56.828708 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a1eafa2f-52d0-415c-8593-82e18693f7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:56.938617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 662/1319] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:46:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:46:56.978361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f51f5c-d342-4f87-a8a8-eeda7f6e2107 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:46:56.979944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-35f51f5c-d342-4f87-a8a8-eeda7f6e2107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:57.092424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f51f5c-d342-4f87-a8a8-eeda7f6e2107 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:46:57.193972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d1ca0-24fe-4445-814f-d0bd517d434a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:46:57.194632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1ca0-24fe-4445-814f-d0bd517d434a service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:46:57.195017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1ca0-24fe-4445-814f-d0bd517d434a service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:46:57.195232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d1ca0-24fe-4445-814f-d0bd517d434a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:46:57.197128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f54d1ca0-24fe-4445-814f-d0bd517d434a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:46:57.335305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d1ca0-24fe-4445-814f-d0bd517d434a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:46:57.337861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d1ca0-24fe-4445-814f-d0bd517d434a service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:46:57.369162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 658/1320] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:46:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:00.588373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507996c4-3e9a-4184-82ba-f278325effed None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:00.588958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507996c4-3e9a-4184-82ba-f278325effed None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:00.588958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507996c4-3e9a-4184-82ba-f278325effed None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:00.590294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 663/1321] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:00.596119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a22a6-e4c9-4fb0-ae13-236210aafebe None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:00.596618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a22a6-e4c9-4fb0-ae13-236210aafebe None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:00.596735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a22a6-e4c9-4fb0-ae13-236210aafebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:00.676801 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-ec0a22a6-e4c9-4fb0-ae13-236210aafebe None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:00.678018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ec0a22a6-e4c9-4fb0-ae13-236210aafebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:00.779116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 659/1322] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:00.832159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb8b02-bf28-4189-94fb-0440d6c2355b None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:00.832440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb8b02-bf28-4189-94fb-0440d6c2355b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:00.832720 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb8b02-bf28-4189-94fb-0440d6c2355b None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:00.834104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 664/1323] 10.222.0.79 () {54 vars in 964 bytes} [Fri May 13 13:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:00.842234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a46f0-2eb9-45df-a09d-eae2b60190d4 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:00.842432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a46f0-2eb9-45df-a09d-eae2b60190d4 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:00.842702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a46f0-2eb9-45df-a09d-eae2b60190d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:00.922080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-566a46f0-2eb9-45df-a09d-eae2b60190d4 None None] MFA Rules not processed for user `8f784cc461ab48ad92140680758a3704`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:00.923298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-566a46f0-2eb9-45df-a09d-eae2b60190d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:01.003040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 660/1324] 10.222.0.79 () {58 vars in 1061 bytes} [Fri May 13 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:01.012995 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb8b02-bf28-4189-94fb-0440d6c2355b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:01.014557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-82eb8b02-bf28-4189-94fb-0440d6c2355b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:01.125512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb8b02-bf28-4189-94fb-0440d6c2355b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:01.215556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68d2f9-ca6f-45a9-8afa-e88ee77ac176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:01.216207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d2f9-ca6f-45a9-8afa-e88ee77ac176 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:01.216484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d2f9-ca6f-45a9-8afa-e88ee77ac176 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:01.216751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d2f9-ca6f-45a9-8afa-e88ee77ac176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:01.218434 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-eb68d2f9-ca6f-45a9-8afa-e88ee77ac176 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:01.342928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68d2f9-ca6f-45a9-8afa-e88ee77ac176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:01.344427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68d2f9-ca6f-45a9-8afa-e88ee77ac176 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:01.365596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 665/1325] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:03.967418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317ebcd-ed79-442d-815c-4a7990e2b463 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:03.968019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317ebcd-ed79-442d-815c-4a7990e2b463 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:03.968019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317ebcd-ed79-442d-815c-4a7990e2b463 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:03.969346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 661/1326] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:03.975710 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3125-55b7-42b0-b4ac-d461d43152cb None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:03.976217 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3125-55b7-42b0-b4ac-d461d43152cb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:03.976298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3125-55b7-42b0-b4ac-d461d43152cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:04.042523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-74dd3125-55b7-42b0-b4ac-d461d43152cb None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:04.043800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-74dd3125-55b7-42b0-b4ac-d461d43152cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:04.148705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 666/1327] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:04.185302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317ebcd-ed79-442d-815c-4a7990e2b463 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:04.186866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4317ebcd-ed79-442d-815c-4a7990e2b463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:04.301468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317ebcd-ed79-442d-815c-4a7990e2b463 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:04.379219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78890a8-caf0-413c-b4eb-543ee00e4715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:04.379874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78890a8-caf0-413c-b4eb-543ee00e4715 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:04.380158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78890a8-caf0-413c-b4eb-543ee00e4715 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:04.380441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78890a8-caf0-413c-b4eb-543ee00e4715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:04.382129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f78890a8-caf0-413c-b4eb-543ee00e4715 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:04.500912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78890a8-caf0-413c-b4eb-543ee00e4715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:04.502282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78890a8-caf0-413c-b4eb-543ee00e4715 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:04.520961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 662/1328] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:06.913389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82f492-0e98-4ca2-bc10-cb5f5b3b75e3 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:06.914488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82f492-0e98-4ca2-bc10-cb5f5b3b75e3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:06.914488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82f492-0e98-4ca2-bc10-cb5f5b3b75e3 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:06.915897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 667/1329] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:06.921673 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bcce3-8ec7-48d7-b65d-2b932351fd8a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:06.922209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bcce3-8ec7-48d7-b65d-2b932351fd8a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:06.922209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bcce3-8ec7-48d7-b65d-2b932351fd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:06.991246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f50bcce3-8ec7-48d7-b65d-2b932351fd8a None None] MFA Rules not processed for user `adb0be7f832d47cfa9b23b4532305e94`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:06.992507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f50bcce3-8ec7-48d7-b65d-2b932351fd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:07.092840 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 663/1330] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:47:07.129928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82f492-0e98-4ca2-bc10-cb5f5b3b75e3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:07.131574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6e82f492-0e98-4ca2-bc10-cb5f5b3b75e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:07.253372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82f492-0e98-4ca2-bc10-cb5f5b3b75e3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:07.334422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cdfc9-00a9-4143-800b-1f2095d1166c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:07.335129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cdfc9-00a9-4143-800b-1f2095d1166c service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:07.335512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cdfc9-00a9-4143-800b-1f2095d1166c service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:07.336330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cdfc9-00a9-4143-800b-1f2095d1166c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:07.338085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-da5cdfc9-00a9-4143-800b-1f2095d1166c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:07.464683 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cdfc9-00a9-4143-800b-1f2095d1166c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:07.466177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cdfc9-00a9-4143-800b-1f2095d1166c service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:07.489043 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 668/1331] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:10.223480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a66e5-836f-4905-8aec-82cbb39378a2 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:10.224016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a66e5-836f-4905-8aec-82cbb39378a2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:10.224016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a66e5-836f-4905-8aec-82cbb39378a2 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:10.225410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 664/1332] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:10.233633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8836d1d-ddfc-4813-9362-6f3f398e2047 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:10.234383 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8836d1d-ddfc-4813-9362-6f3f398e2047 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:10.234383 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8836d1d-ddfc-4813-9362-6f3f398e2047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:10.325764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a8836d1d-ddfc-4813-9362-6f3f398e2047 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:10.329557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a8836d1d-ddfc-4813-9362-6f3f398e2047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:10.459225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 669/1333] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:10.498400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a66e5-836f-4905-8aec-82cbb39378a2 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:10.500031 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b26a66e5-836f-4905-8aec-82cbb39378a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:10.638489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a66e5-836f-4905-8aec-82cbb39378a2 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:10.729832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380ba89-087c-4d71-bc49-b5ccb7e46199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:10.730382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380ba89-087c-4d71-bc49-b5ccb7e46199 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:10.730658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380ba89-087c-4d71-bc49-b5ccb7e46199 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:10.731328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380ba89-087c-4d71-bc49-b5ccb7e46199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:10.732605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e380ba89-087c-4d71-bc49-b5ccb7e46199 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:10.856622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380ba89-087c-4d71-bc49-b5ccb7e46199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:10.857977 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380ba89-087c-4d71-bc49-b5ccb7e46199 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:10.876227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 665/1334] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:13.224471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234511d-512d-4d84-8962-6f6e7cd104af None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:13.225012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234511d-512d-4d84-8962-6f6e7cd104af None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:13.225012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234511d-512d-4d84-8962-6f6e7cd104af None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:13.226986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 670/1335] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:13.232630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad0c09-6d4c-4d48-9962-3d8ac55cea7e None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:13.233155 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad0c09-6d4c-4d48-9962-3d8ac55cea7e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:13.233155 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad0c09-6d4c-4d48-9962-3d8ac55cea7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:13.303001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-12ad0c09-6d4c-4d48-9962-3d8ac55cea7e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:13.304261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-12ad0c09-6d4c-4d48-9962-3d8ac55cea7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:13.405111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 666/1336] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:13.446729 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234511d-512d-4d84-8962-6f6e7cd104af None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:13.448289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c234511d-512d-4d84-8962-6f6e7cd104af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:13.566477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234511d-512d-4d84-8962-6f6e7cd104af None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:13.641676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47718c2d-715d-4431-adfd-0c2c9058c970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:13.642320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47718c2d-715d-4431-adfd-0c2c9058c970 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:13.642595 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47718c2d-715d-4431-adfd-0c2c9058c970 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:13.642895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47718c2d-715d-4431-adfd-0c2c9058c970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:13.644534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-47718c2d-715d-4431-adfd-0c2c9058c970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:13.755839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47718c2d-715d-4431-adfd-0c2c9058c970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:13.757292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47718c2d-715d-4431-adfd-0c2c9058c970 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:13.775830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 671/1337] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:16.426316 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1d21d-1f12-4203-a02c-12c22222b31a None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:16.427218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1d21d-1f12-4203-a02c-12c22222b31a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:16.427218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1d21d-1f12-4203-a02c-12c22222b31a None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:16.428681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 667/1338] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:16.435133 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b8747-fbf7-4d2e-bf51-1cd5c6eae370 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:16.435706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b8747-fbf7-4d2e-bf51-1cd5c6eae370 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:16.435706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b8747-fbf7-4d2e-bf51-1cd5c6eae370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:16.509815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-914b8747-fbf7-4d2e-bf51-1cd5c6eae370 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:16.510982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-914b8747-fbf7-4d2e-bf51-1cd5c6eae370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:16.622281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 672/1339] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:47:16.657864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1d21d-1f12-4203-a02c-12c22222b31a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:16.660128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a9a1d21d-1f12-4203-a02c-12c22222b31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:16.766697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1d21d-1f12-4203-a02c-12c22222b31a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:16.770218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d712e04-606b-4a7f-b6a2-257505d44faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:16.770896 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d712e04-606b-4a7f-b6a2-257505d44faa service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:16.771189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d712e04-606b-4a7f-b6a2-257505d44faa service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:16.771452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d712e04-606b-4a7f-b6a2-257505d44faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:16.781028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d712e04-606b-4a7f-b6a2-257505d44faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:16.782481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d712e04-606b-4a7f-b6a2-257505d44faa service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:16.792815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 668/1340] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:21.206861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12119cba-c767-475e-a37a-fb362de4a2c7 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:21.207435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12119cba-c767-475e-a37a-fb362de4a2c7 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:21.207435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12119cba-c767-475e-a37a-fb362de4a2c7 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:21.208778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 673/1341] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:21.215478 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192eb5b-899b-422f-8813-50257826def5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:21.216202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192eb5b-899b-422f-8813-50257826def5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:21.216202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192eb5b-899b-422f-8813-50257826def5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:21.256838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-9192eb5b-899b-422f-8813-50257826def5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:21.258013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9192eb5b-899b-422f-8813-50257826def5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:21.345526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 669/1342] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:47:21.385204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12119cba-c767-475e-a37a-fb362de4a2c7 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:21.396325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12119cba-c767-475e-a37a-fb362de4a2c7 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:21.401141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef1890-9bd8-41e1-9ce7-7a0da8201d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:21.402210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef1890-9bd8-41e1-9ce7-7a0da8201d7e service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:21.402697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef1890-9bd8-41e1-9ce7-7a0da8201d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:21.403008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef1890-9bd8-41e1-9ce7-7a0da8201d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:21.416672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef1890-9bd8-41e1-9ce7-7a0da8201d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:21.418118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef1890-9bd8-41e1-9ce7-7a0da8201d7e service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:21.428621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 674/1343] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:24.391347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df45761-d0a2-4d4a-9bd8-45e71472f65b None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:24.391907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df45761-d0a2-4d4a-9bd8-45e71472f65b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:24.391907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df45761-d0a2-4d4a-9bd8-45e71472f65b None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:24.393310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 670/1344] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:24.399390 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93f062-1b1c-4bd9-b15c-a7eb34871ea1 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:24.399949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93f062-1b1c-4bd9-b15c-a7eb34871ea1 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:24.400046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93f062-1b1c-4bd9-b15c-a7eb34871ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:24.418426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-9c93f062-1b1c-4bd9-b15c-a7eb34871ea1 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:24.419679 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9c93f062-1b1c-4bd9-b15c-a7eb34871ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:24.427597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 675/1345] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:24.464737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df45761-d0a2-4d4a-9bd8-45e71472f65b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:24.466557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7df45761-d0a2-4d4a-9bd8-45e71472f65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:24.480234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df45761-d0a2-4d4a-9bd8-45e71472f65b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:24.483887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d547fb-1d0e-4b5e-96ac-a9cd3a600fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:24.484533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d547fb-1d0e-4b5e-96ac-a9cd3a600fff service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:24.484814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d547fb-1d0e-4b5e-96ac-a9cd3a600fff service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:24.485088 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d547fb-1d0e-4b5e-96ac-a9cd3a600fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:24.495629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d547fb-1d0e-4b5e-96ac-a9cd3a600fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:24.497023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d547fb-1d0e-4b5e-96ac-a9cd3a600fff service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:24.506887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 671/1346] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:29.658472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc544109-62a4-4fd5-ad8f-adfcf600490c None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:29.659013 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc544109-62a4-4fd5-ad8f-adfcf600490c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:29.659013 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc544109-62a4-4fd5-ad8f-adfcf600490c None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:29.660408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 676/1347] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:29.668284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79712d0-c7c8-4499-99e0-cd2e05fa39dd None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:29.668814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79712d0-c7c8-4499-99e0-cd2e05fa39dd None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:29.668814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79712d0-c7c8-4499-99e0-cd2e05fa39dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:29.687532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f79712d0-c7c8-4499-99e0-cd2e05fa39dd None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:29.688747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f79712d0-c7c8-4499-99e0-cd2e05fa39dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:29.697278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 672/1348] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:29.734822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc544109-62a4-4fd5-ad8f-adfcf600490c None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:29.746186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc544109-62a4-4fd5-ad8f-adfcf600490c None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:29.749676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663edbe0-bb31-4e8e-9154-35f20e586720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:29.750317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663edbe0-bb31-4e8e-9154-35f20e586720 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:29.750601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663edbe0-bb31-4e8e-9154-35f20e586720 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:29.750918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663edbe0-bb31-4e8e-9154-35f20e586720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:29.760346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663edbe0-bb31-4e8e-9154-35f20e586720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:29.761797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663edbe0-bb31-4e8e-9154-35f20e586720 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:29.771972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 677/1349] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:35.481802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea82ede-7498-49f3-89cf-bafdb663ebe0 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:35.484412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea82ede-7498-49f3-89cf-bafdb663ebe0 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:35.484951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea82ede-7498-49f3-89cf-bafdb663ebe0 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:35.486667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 673/1350] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 13:47:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:35.492622 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d0b59-5d13-4b17-b7d9-e64bc7a30534 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:35.494028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d0b59-5d13-4b17-b7d9-e64bc7a30534 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:35.494822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d0b59-5d13-4b17-b7d9-e64bc7a30534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:35.671697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-cc5d0b59-5d13-4b17-b7d9-e64bc7a30534 None None] MFA Rules not processed for user `4ae9a6f1ac304e21b8d3115f2a0baf88`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:35.673021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-cc5d0b59-5d13-4b17-b7d9-e64bc7a30534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:35.812619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 678/1351] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 13:47:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:40.727810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b7465-6f1f-4a91-a6af-32a279a6bb92 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:40.728505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b7465-6f1f-4a91-a6af-32a279a6bb92 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:40.729130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b7465-6f1f-4a91-a6af-32a279a6bb92 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:40.732039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 674/1352] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 13:47:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:40.737265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f581668-3df1-41fc-82c9-ee59ce7874cf None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:40.737791 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f581668-3df1-41fc-82c9-ee59ce7874cf None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:40.737908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f581668-3df1-41fc-82c9-ee59ce7874cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:40.757649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6f581668-3df1-41fc-82c9-ee59ce7874cf None None] MFA Rules not processed for user `4ae9a6f1ac304e21b8d3115f2a0baf88`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:40.758908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6f581668-3df1-41fc-82c9-ee59ce7874cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:40.768026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 679/1353] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 13:47:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:42.571565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c6dd3-2267-4290-bd6b-bf800d36e28e None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:42.572067 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c6dd3-2267-4290-bd6b-bf800d36e28e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:42.572067 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c6dd3-2267-4290-bd6b-bf800d36e28e None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:42.574935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 675/1354] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 13:47:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:42.579254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9cc5b-a92d-4bc2-ba74-9ebd434c5cd6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:42.579924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9cc5b-a92d-4bc2-ba74-9ebd434c5cd6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:42.579924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9cc5b-a92d-4bc2-ba74-9ebd434c5cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:42.599963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0fd9cc5b-a92d-4bc2-ba74-9ebd434c5cd6 None None] MFA Rules not processed for user `4ae9a6f1ac304e21b8d3115f2a0baf88`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:42.601210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0fd9cc5b-a92d-4bc2-ba74-9ebd434c5cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:42.618000 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 680/1355] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 13:47:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:46.988854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084033f-3e16-4cc1-b948-ed711934faf5 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:46.989867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084033f-3e16-4cc1-b948-ed711934faf5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:46.990308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084033f-3e16-4cc1-b948-ed711934faf5 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:46.993842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 676/1356] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:47.001708 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e534d7-9aef-4e90-b1e7-74ba5c7528e4 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:47.002295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e534d7-9aef-4e90-b1e7-74ba5c7528e4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:47.002295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e534d7-9aef-4e90-b1e7-74ba5c7528e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:47.022846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a3e534d7-9aef-4e90-b1e7-74ba5c7528e4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:47.024056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a3e534d7-9aef-4e90-b1e7-74ba5c7528e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:47.032214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 681/1357] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:47.084629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb609343-295c-47f9-a305-a7764122213c None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:47.084847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb609343-295c-47f9-a305-a7764122213c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:47.085096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb609343-295c-47f9-a305-a7764122213c None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:47.086523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 677/1358] 10.222.0.79 () {54 vars in 961 bytes} [Fri May 13 13:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:47:47.093025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9610b2-e0a3-414f-bc22-15fa4ecdb1c9 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:47.093298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9610b2-e0a3-414f-bc22-15fa4ecdb1c9 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:47.093574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9610b2-e0a3-414f-bc22-15fa4ecdb1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:47.153529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-bf9610b2-e0a3-414f-bc22-15fa4ecdb1c9 None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:47.155040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-bf9610b2-e0a3-414f-bc22-15fa4ecdb1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:47.247781 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 682/1359] 10.222.0.79 () {58 vars in 1058 bytes} [Fri May 13 13:47:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:47.255961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb609343-295c-47f9-a305-a7764122213c None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:47.270636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb609343-295c-47f9-a305-a7764122213c None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:47.280452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb2763-ec7c-4d4d-8dfc-8e871974c37f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:47.281379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb2763-ec7c-4d4d-8dfc-8e871974c37f service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:47.281656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb2763-ec7c-4d4d-8dfc-8e871974c37f service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:47.281929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb2763-ec7c-4d4d-8dfc-8e871974c37f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:47.291582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb2763-ec7c-4d4d-8dfc-8e871974c37f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:47.293327 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb2763-ec7c-4d4d-8dfc-8e871974c37f service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:47.303433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 678/1360] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:49.830054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d056e2-974f-4e94-af57-86bb29943e8d None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:49.830695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d056e2-974f-4e94-af57-86bb29943e8d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:49.830695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d056e2-974f-4e94-af57-86bb29943e8d None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:49.832389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 683/1361] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:49.838884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d80d3-eb20-4650-a7c2-38346de1c73f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:49.839449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d80d3-eb20-4650-a7c2-38346de1c73f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:49.839449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d80d3-eb20-4650-a7c2-38346de1c73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:49.859494 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-908d80d3-eb20-4650-a7c2-38346de1c73f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:49.859686 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-908d80d3-eb20-4650-a7c2-38346de1c73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:49.867745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 679/1362] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:49.913122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d056e2-974f-4e94-af57-86bb29943e8d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:49.940169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d056e2-974f-4e94-af57-86bb29943e8d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:49.945311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446b5f1-9401-4efe-8981-30881710b42f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:49.946041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446b5f1-9401-4efe-8981-30881710b42f service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:49.946395 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446b5f1-9401-4efe-8981-30881710b42f service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:49.946938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446b5f1-9401-4efe-8981-30881710b42f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:49.956531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446b5f1-9401-4efe-8981-30881710b42f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:49.958534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446b5f1-9401-4efe-8981-30881710b42f service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:49.969517 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 684/1363] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:52.744249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529ba37-dcf3-40ce-b00e-8197f0dd43d8 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:52.744816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529ba37-dcf3-40ce-b00e-8197f0dd43d8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:52.744816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529ba37-dcf3-40ce-b00e-8197f0dd43d8 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:52.746420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 680/1364] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:52.752754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b0007-d7f4-4d5b-8b5d-db0f82586955 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:52.755165 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b0007-d7f4-4d5b-8b5d-db0f82586955 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:52.755165 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b0007-d7f4-4d5b-8b5d-db0f82586955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:52.779359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-9d1b0007-d7f4-4d5b-8b5d-db0f82586955 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:52.780569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9d1b0007-d7f4-4d5b-8b5d-db0f82586955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:52.788806 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 685/1365] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:52.839227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246fee9-171e-438d-88e1-12543f204e52 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:52.839513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246fee9-171e-438d-88e1-12543f204e52 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:52.839792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246fee9-171e-438d-88e1-12543f204e52 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:52.841185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 681/1366] 10.222.0.79 () {54 vars in 961 bytes} [Fri May 13 13:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:52.847097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10eb07-e895-4a11-96b3-f3c67ee84853 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:52.847482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10eb07-e895-4a11-96b3-f3c67ee84853 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:52.847757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10eb07-e895-4a11-96b3-f3c67ee84853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:52.868337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-4a10eb07-e895-4a11-96b3-f3c67ee84853 None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:52.869667 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-4a10eb07-e895-4a11-96b3-f3c67ee84853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:52.880566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 686/1367] 10.222.0.79 () {58 vars in 1058 bytes} [Fri May 13 13:47:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:52.889203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246fee9-171e-438d-88e1-12543f204e52 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:52.904542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246fee9-171e-438d-88e1-12543f204e52 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:52.910413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd34fd-4292-4299-afda-aeb599c4ee94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:52.911114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd34fd-4292-4299-afda-aeb599c4ee94 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:52.911391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd34fd-4292-4299-afda-aeb599c4ee94 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:52.911663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd34fd-4292-4299-afda-aeb599c4ee94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:52.921621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd34fd-4292-4299-afda-aeb599c4ee94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:52.923790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd34fd-4292-4299-afda-aeb599c4ee94 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:52.940108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 682/1368] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:47:55.601476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a32601-14e6-4a00-ac85-e20b5a76b9d0 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:55.603285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a32601-14e6-4a00-ac85-e20b5a76b9d0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:55.603864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a32601-14e6-4a00-ac85-e20b5a76b9d0 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:55.606432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 687/1369] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:55.614914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5356-674f-41d9-9f5c-205b1912f3ec None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:55.615623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5356-674f-41d9-9f5c-205b1912f3ec None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:55.615904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5356-674f-41d9-9f5c-205b1912f3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:55.645323 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c7ed5356-674f-41d9-9f5c-205b1912f3ec None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:55.646590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c7ed5356-674f-41d9-9f5c-205b1912f3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:55.656039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 683/1370] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:55.717714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a32601-14e6-4a00-ac85-e20b5a76b9d0 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:55.739965 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a32601-14e6-4a00-ac85-e20b5a76b9d0 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:55.746881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a963b9-3b4d-4cfd-8b9e-dafe961082fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:55.747726 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a963b9-3b4d-4cfd-8b9e-dafe961082fe service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:55.748113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a963b9-3b4d-4cfd-8b9e-dafe961082fe service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:55.748486 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a963b9-3b4d-4cfd-8b9e-dafe961082fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:55.760228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a963b9-3b4d-4cfd-8b9e-dafe961082fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:55.762441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a963b9-3b4d-4cfd-8b9e-dafe961082fe service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:55.777776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 688/1371] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:47:56.302550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6a9f7-77fa-490d-8b53-bb54b5927083 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:56.303415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6a9f7-77fa-490d-8b53-bb54b5927083 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:56.303941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6a9f7-77fa-490d-8b53-bb54b5927083 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:56.306528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 684/1372] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:56.313139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6e84d-5995-4063-912b-d83d9a42450c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:56.313501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6e84d-5995-4063-912b-d83d9a42450c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:56.313709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6e84d-5995-4063-912b-d83d9a42450c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:56.335885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-d3d6e84d-5995-4063-912b-d83d9a42450c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:56.337386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d3d6e84d-5995-4063-912b-d83d9a42450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:56.346588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 689/1373] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:56.645488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22016777-6ae8-4816-89c0-5ac027eaa283 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:56.646157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22016777-6ae8-4816-89c0-5ac027eaa283 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:56.646157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22016777-6ae8-4816-89c0-5ac027eaa283 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:56.647463 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 685/1374] 10.222.0.79 () {54 vars in 963 bytes} [Fri May 13 13:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:56.653535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436adf-9ec6-439d-adb6-6572ff8e6431 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:56.654364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436adf-9ec6-439d-adb6-6572ff8e6431 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:56.654364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436adf-9ec6-439d-adb6-6572ff8e6431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:56.716412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-71436adf-9ec6-439d-adb6-6572ff8e6431 None None] MFA Rules not processed for user `e03fa33a68ff4567bc25c9611cd9ae8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:56.717282 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-71436adf-9ec6-439d-adb6-6572ff8e6431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:56.795185 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 690/1375] 10.222.0.79 () {58 vars in 1060 bytes} [Fri May 13 13:47:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:56.803393 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22016777-6ae8-4816-89c0-5ac027eaa283 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:56.816360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22016777-6ae8-4816-89c0-5ac027eaa283 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:56.820634 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e927fa-b471-45bd-ac4b-03504f9a8185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:56.821415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e927fa-b471-45bd-ac4b-03504f9a8185 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:56.821757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e927fa-b471-45bd-ac4b-03504f9a8185 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:56.822162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e927fa-b471-45bd-ac4b-03504f9a8185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:56.833782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e927fa-b471-45bd-ac4b-03504f9a8185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:56.836075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e927fa-b471-45bd-ac4b-03504f9a8185 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:56.847859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 686/1376] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:47:58.357482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603cc8c-37e5-4469-9a33-0643d9f70b87 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:58.357482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603cc8c-37e5-4469-9a33-0643d9f70b87 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:58.357482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603cc8c-37e5-4469-9a33-0643d9f70b87 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:58.359052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 691/1377] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:47:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:47:58.364361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81cae6-c507-419b-b25c-3fa5b69497e3 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:58.365170 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81cae6-c507-419b-b25c-3fa5b69497e3 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:58.365538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81cae6-c507-419b-b25c-3fa5b69497e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:58.385221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-db81cae6-c507-419b-b25c-3fa5b69497e3 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:47:58.386424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-db81cae6-c507-419b-b25c-3fa5b69497e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:47:58.395110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 687/1378] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:47:58.441587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603cc8c-37e5-4469-9a33-0643d9f70b87 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:47:58.456907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603cc8c-37e5-4469-9a33-0643d9f70b87 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:47:58.463491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761f05b-0e10-4237-9ef1-25f1b0b106d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:47:58.464140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761f05b-0e10-4237-9ef1-25f1b0b106d1 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:47:58.464415 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761f05b-0e10-4237-9ef1-25f1b0b106d1 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:47:58.464683 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761f05b-0e10-4237-9ef1-25f1b0b106d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:47:58.473774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761f05b-0e10-4237-9ef1-25f1b0b106d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:47:58.475621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761f05b-0e10-4237-9ef1-25f1b0b106d1 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:47:58.486544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 692/1379] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:47:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:00.881981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66983abd-f86a-4672-8b32-7d6f1e2fa3f6 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:00.883159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66983abd-f86a-4672-8b32-7d6f1e2fa3f6 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:00.883159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66983abd-f86a-4672-8b32-7d6f1e2fa3f6 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:00.884921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 688/1380] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:00.890927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff525c-4fed-40cf-8b0b-c6573ed22d76 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:00.891801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff525c-4fed-40cf-8b0b-c6573ed22d76 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:00.892249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff525c-4fed-40cf-8b0b-c6573ed22d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:00.923394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-95ff525c-4fed-40cf-8b0b-c6573ed22d76 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:00.924891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-95ff525c-4fed-40cf-8b0b-c6573ed22d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:00.940086 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 693/1381] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:01.003447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66983abd-f86a-4672-8b32-7d6f1e2fa3f6 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:01.020279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66983abd-f86a-4672-8b32-7d6f1e2fa3f6 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:01.026575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dae3f2-d4a8-4b08-95fa-8d42792102cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:01.027432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dae3f2-d4a8-4b08-95fa-8d42792102cd service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:01.028053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dae3f2-d4a8-4b08-95fa-8d42792102cd service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:01.028390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dae3f2-d4a8-4b08-95fa-8d42792102cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:01.037899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dae3f2-d4a8-4b08-95fa-8d42792102cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:01.041158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dae3f2-d4a8-4b08-95fa-8d42792102cd service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:01.057061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 689/1382] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:03.500119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce51a0-5c69-4d6f-afc5-cd49c4acc1c7 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:03.500559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce51a0-5c69-4d6f-afc5-cd49c4acc1c7 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:03.500658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce51a0-5c69-4d6f-afc5-cd49c4acc1c7 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:03.502231 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 694/1383] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:48:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:03.508635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db02a5-1e04-4475-b102-1d97f4755d8f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:03.509068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db02a5-1e04-4475-b102-1d97f4755d8f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:03.509068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db02a5-1e04-4475-b102-1d97f4755d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:03.536310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-58db02a5-1e04-4475-b102-1d97f4755d8f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:03.538583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-58db02a5-1e04-4475-b102-1d97f4755d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:03.549664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 690/1384] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:03.594567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce51a0-5c69-4d6f-afc5-cd49c4acc1c7 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:03.610386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce51a0-5c69-4d6f-afc5-cd49c4acc1c7 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:03.618050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15e839-3651-4bcb-80ed-2d87c08ba085 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:03.618917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e839-3651-4bcb-80ed-2d87c08ba085 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:03.619266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e839-3651-4bcb-80ed-2d87c08ba085 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:03.619611 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e839-3651-4bcb-80ed-2d87c08ba085 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:03.628974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15e839-3651-4bcb-80ed-2d87c08ba085 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:03.630919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15e839-3651-4bcb-80ed-2d87c08ba085 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:03.642430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 695/1385] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:48:06.281999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb567-4fb2-447c-8ac2-495e3d8d9883 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:06.282993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb567-4fb2-447c-8ac2-495e3d8d9883 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:06.282993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb567-4fb2-447c-8ac2-495e3d8d9883 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:06.284486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 691/1386] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:06.293923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad397-3068-4078-9a05-e56c2e749419 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:06.294474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad397-3068-4078-9a05-e56c2e749419 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:06.294657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad397-3068-4078-9a05-e56c2e749419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:06.322564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b72ad397-3068-4078-9a05-e56c2e749419 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:06.324145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b72ad397-3068-4078-9a05-e56c2e749419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:06.333167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 696/1387] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:06.380928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fb567-4fb2-447c-8ac2-495e3d8d9883 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:06.402032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fb567-4fb2-447c-8ac2-495e3d8d9883 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:06.409164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5847694-fa07-4b55-90d7-806e1fa37dd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:06.409979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5847694-fa07-4b55-90d7-806e1fa37dd8 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:06.410380 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5847694-fa07-4b55-90d7-806e1fa37dd8 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:06.410878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5847694-fa07-4b55-90d7-806e1fa37dd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:06.426020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5847694-fa07-4b55-90d7-806e1fa37dd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:06.428221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5847694-fa07-4b55-90d7-806e1fa37dd8 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:06.441396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 692/1388] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:48:09.230258 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d47289-2af1-4446-b2ad-3b5349862ea2 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:09.231011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d47289-2af1-4446-b2ad-3b5349862ea2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:09.231011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d47289-2af1-4446-b2ad-3b5349862ea2 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:09.232354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 697/1389] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:48:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:09.244135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a19242-696a-4b63-85b0-089b17dc4250 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:09.244722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a19242-696a-4b63-85b0-089b17dc4250 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:09.245358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a19242-696a-4b63-85b0-089b17dc4250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:09.281598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-29a19242-696a-4b63-85b0-089b17dc4250 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:09.283335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-29a19242-696a-4b63-85b0-089b17dc4250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:09.294397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 693/1390] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:09.349909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d47289-2af1-4446-b2ad-3b5349862ea2 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:09.368521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d47289-2af1-4446-b2ad-3b5349862ea2 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:09.376916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ee9d0-de37-4cb6-8911-06dc5efe311d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:09.377880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ee9d0-de37-4cb6-8911-06dc5efe311d service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:09.378197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ee9d0-de37-4cb6-8911-06dc5efe311d service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:09.378671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ee9d0-de37-4cb6-8911-06dc5efe311d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:09.392871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ee9d0-de37-4cb6-8911-06dc5efe311d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:09.395442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ee9d0-de37-4cb6-8911-06dc5efe311d service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:09.409973 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 698/1391] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:12.153062 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d530f0-edf8-430d-a790-daf0453b1816 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:12.153670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d530f0-edf8-430d-a790-daf0453b1816 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:12.153670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d530f0-edf8-430d-a790-daf0453b1816 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:12.155141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 694/1392] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:12.161168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48031c-2cb7-4a2b-befb-0764e661a6cb None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:12.161766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48031c-2cb7-4a2b-befb-0764e661a6cb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:12.161863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48031c-2cb7-4a2b-befb-0764e661a6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:12.181676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-5c48031c-2cb7-4a2b-befb-0764e661a6cb None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:12.182979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5c48031c-2cb7-4a2b-befb-0764e661a6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:12.191286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 699/1393] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:48:12.248780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d530f0-edf8-430d-a790-daf0453b1816 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:12.265815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d530f0-edf8-430d-a790-daf0453b1816 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:12.277586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24808a7f-d9c1-4ade-b2a6-b48d255097bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:12.278256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808a7f-d9c1-4ade-b2a6-b48d255097bb service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:12.278541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808a7f-d9c1-4ade-b2a6-b48d255097bb service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:12.278857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808a7f-d9c1-4ade-b2a6-b48d255097bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:12.300652 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24808a7f-d9c1-4ade-b2a6-b48d255097bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:12.304823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24808a7f-d9c1-4ade-b2a6-b48d255097bb service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:12.327407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 695/1394] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:48:14.795118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276ee3e-5862-47b1-b294-f62f4693488f None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:14.796030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276ee3e-5862-47b1-b294-f62f4693488f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:14.796030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276ee3e-5862-47b1-b294-f62f4693488f None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:14.797056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 700/1395] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:14.802782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8b9a2-a274-40a1-b2f6-e4612891b572 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:14.803317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8b9a2-a274-40a1-b2f6-e4612891b572 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:14.803438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8b9a2-a274-40a1-b2f6-e4612891b572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:14.822627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c6b8b9a2-a274-40a1-b2f6-e4612891b572 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:14.824587 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c6b8b9a2-a274-40a1-b2f6-e4612891b572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:14.846384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 696/1396] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:14.894034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276ee3e-5862-47b1-b294-f62f4693488f None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:14.911703 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276ee3e-5862-47b1-b294-f62f4693488f None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:14.916597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe203e72-3266-43e0-b92d-9f9a5303c300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:14.917247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe203e72-3266-43e0-b92d-9f9a5303c300 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:14.917524 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe203e72-3266-43e0-b92d-9f9a5303c300 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:14.917792 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe203e72-3266-43e0-b92d-9f9a5303c300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:14.927102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe203e72-3266-43e0-b92d-9f9a5303c300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:14.929329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe203e72-3266-43e0-b92d-9f9a5303c300 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:14.940021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 701/1397] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:15.068647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c05c95-aa6c-4ade-b2ec-be2b98fb46b4 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:15.068850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c05c95-aa6c-4ade-b2ec-be2b98fb46b4 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:15.068967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c05c95-aa6c-4ade-b2ec-be2b98fb46b4 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:15.070440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 697/1398] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:15.076607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121009c6-c67e-47c1-91d0-14a51346774e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:15.076891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121009c6-c67e-47c1-91d0-14a51346774e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:15.077164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121009c6-c67e-47c1-91d0-14a51346774e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:15.097233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-121009c6-c67e-47c1-91d0-14a51346774e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:15.098452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-121009c6-c67e-47c1-91d0-14a51346774e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:15.107503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 702/1399] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:15.167376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4e42f-19ce-4cdf-9960-97a71237f68b None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:15.167773 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4e42f-19ce-4cdf-9960-97a71237f68b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:15.168118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4e42f-19ce-4cdf-9960-97a71237f68b None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:15.169738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 698/1400] 10.222.0.79 () {54 vars in 962 bytes} [Fri May 13 13:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:15.176561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66896d-4cb0-4c04-b4e3-7e4f65d6c802 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:15.176890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66896d-4cb0-4c04-b4e3-7e4f65d6c802 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:15.177145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66896d-4cb0-4c04-b4e3-7e4f65d6c802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:15.227357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-be66896d-4cb0-4c04-b4e3-7e4f65d6c802 None None] MFA Rules not processed for user `cc4f45e859d84ceaa5a01477e83e098e`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:15.228568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-be66896d-4cb0-4c04-b4e3-7e4f65d6c802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:15.290184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 703/1401] 10.222.0.79 () {58 vars in 1059 bytes} [Fri May 13 13:48:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:15.297818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4e42f-19ce-4cdf-9960-97a71237f68b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:15.309909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4e42f-19ce-4cdf-9960-97a71237f68b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:15.313556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9aee04-8427-4ef8-b64b-1c2c3c362938 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4f45e859d84ceaa5a01477e83e098e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:15.314185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aee04-8427-4ef8-b64b-1c2c3c362938 service glance] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:15.314474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aee04-8427-4ef8-b64b-1c2c3c362938 service glance] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:15.314766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aee04-8427-4ef8-b64b-1c2c3c362938 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:15.325160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9aee04-8427-4ef8-b64b-1c2c3c362938 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:15.326524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9aee04-8427-4ef8-b64b-1c2c3c362938 service glance] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:15.337306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 699/1402] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:15.763556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201defd-6e51-4990-9a94-c96285a9454e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:15.764011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201defd-6e51-4990-9a94-c96285a9454e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:15.764351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201defd-6e51-4990-9a94-c96285a9454e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:15.811954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-7201defd-6e51-4990-9a94-c96285a9454e None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:15.813147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7201defd-6e51-4990-9a94-c96285a9454e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:15.878230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 704/1403] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 13:48:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:15.919573 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20274-7ff3-41c0-8117-020a002df9d8 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:15.920141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20274-7ff3-41c0-8117-020a002df9d8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:15.920270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20274-7ff3-41c0-8117-020a002df9d8 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:15.921556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 700/1404] 10.222.0.79 () {54 vars in 942 bytes} [Fri May 13 13:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:15.928243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593e444-5bb0-42e0-b79e-2454c49e6d5e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:15.929036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593e444-5bb0-42e0-b79e-2454c49e6d5e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:15.929336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593e444-5bb0-42e0-b79e-2454c49e6d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:15.978330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2593e444-5bb0-42e0-b79e-2454c49e6d5e None None] MFA Rules not processed for user `4139cfe906bd4c7d94226ab589577627`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:15.979577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2593e444-5bb0-42e0-b79e-2454c49e6d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:16.049289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 705/1405] 10.222.0.79 () {58 vars in 1039 bytes} [Fri May 13 13:48:15 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:16.059791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d20274-7ff3-41c0-8117-020a002df9d8 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:16.079671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d20274-7ff3-41c0-8117-020a002df9d8 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:16.085157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724ee4d-b62f-4d6a-a828-ce1a41f071c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:16.086555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724ee4d-b62f-4d6a-a828-ce1a41f071c6 service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:16.086555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724ee4d-b62f-4d6a-a828-ce1a41f071c6 service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:16.086555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724ee4d-b62f-4d6a-a828-ce1a41f071c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:16.096110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724ee4d-b62f-4d6a-a828-ce1a41f071c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:16.097728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724ee4d-b62f-4d6a-a828-ce1a41f071c6 service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:16.106637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 701/1406] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 13:48:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:48:17.305381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49098d-b8c7-4a19-b285-c4aa13456a6a None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:17.305942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49098d-b8c7-4a19-b285-c4aa13456a6a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:17.305942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49098d-b8c7-4a19-b285-c4aa13456a6a None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:17.307387 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 706/1407] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:48:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:17.313194 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f414f17-af95-44ff-9f59-49264582c42b None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:17.314049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f414f17-af95-44ff-9f59-49264582c42b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:17.314354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f414f17-af95-44ff-9f59-49264582c42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:17.335066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1f414f17-af95-44ff-9f59-49264582c42b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:17.336461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1f414f17-af95-44ff-9f59-49264582c42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:17.345139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 702/1408] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:48:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:17.392758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49098d-b8c7-4a19-b285-c4aa13456a6a None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:17.427020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49098d-b8c7-4a19-b285-c4aa13456a6a None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:17.437007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b60a8-a241-4891-93c6-32a92deab9ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:17.438220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b60a8-a241-4891-93c6-32a92deab9ee service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:17.438798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b60a8-a241-4891-93c6-32a92deab9ee service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:17.439232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b60a8-a241-4891-93c6-32a92deab9ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:17.451209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b60a8-a241-4891-93c6-32a92deab9ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:17.453365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b60a8-a241-4891-93c6-32a92deab9ee service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:17.466411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 707/1409] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:40.434647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009f998-a517-49c4-9748-8370209178b9 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:40.435304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009f998-a517-49c4-9748-8370209178b9 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:40.435304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009f998-a517-49c4-9748-8370209178b9 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:40.436606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 703/1410] 10.222.0.79 () {54 vars in 918 bytes} [Fri May 13 13:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:40.442189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077b2c1-deb2-4eb9-bf40-44ac12f82226 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:40.442771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077b2c1-deb2-4eb9-bf40-44ac12f82226 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:40.442771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077b2c1-deb2-4eb9-bf40-44ac12f82226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:40.462065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0077b2c1-deb2-4eb9-bf40-44ac12f82226 None None] MFA Rules not processed for user `4ae9a6f1ac304e21b8d3115f2a0baf88`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:40.463330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0077b2c1-deb2-4eb9-bf40-44ac12f82226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:40.471961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 708/1411] 10.222.0.79 () {58 vars in 1015 bytes} [Fri May 13 13:48:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:40.486624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28213df-281e-4561-92a1-e3b959a66561 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:40.486936 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28213df-281e-4561-92a1-e3b959a66561 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:40.487217 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28213df-281e-4561-92a1-e3b959a66561 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:40.488599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 704/1412] 10.222.0.79 () {54 vars in 918 bytes} [Fri May 13 13:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:40.497060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c18b7d-1cb1-4f2d-895c-610476c03d59 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:40.497342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c18b7d-1cb1-4f2d-895c-610476c03d59 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:40.497619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c18b7d-1cb1-4f2d-895c-610476c03d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:40.516116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-29c18b7d-1cb1-4f2d-895c-610476c03d59 None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:40.517343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-29c18b7d-1cb1-4f2d-895c-610476c03d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:40.526756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 709/1413] 10.222.0.79 () {58 vars in 1015 bytes} [Fri May 13 13:48:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:43.336148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2544ea-fdd5-4759-93ff-fcac9212be2a None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:43.336714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2544ea-fdd5-4759-93ff-fcac9212be2a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:43.336714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2544ea-fdd5-4759-93ff-fcac9212be2a None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:43.337996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:43.343840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16257571-ff84-46cc-8cc3-5c3b64c32a69 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:43.344303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16257571-ff84-46cc-8cc3-5c3b64c32a69 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:43.344427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16257571-ff84-46cc-8cc3-5c3b64c32a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:43.365209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-16257571-ff84-46cc-8cc3-5c3b64c32a69 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:43.367958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-16257571-ff84-46cc-8cc3-5c3b64c32a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:43.381154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:43.428124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2544ea-fdd5-4759-93ff-fcac9212be2a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:43.444642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2544ea-fdd5-4759-93ff-fcac9212be2a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:43.450419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f10e0-10dc-4e93-ad8b-fef030cbd02a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:43.451113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f10e0-10dc-4e93-ad8b-fef030cbd02a service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:43.451413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f10e0-10dc-4e93-ad8b-fef030cbd02a service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:43.451681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f10e0-10dc-4e93-ad8b-fef030cbd02a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:43.461516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f10e0-10dc-4e93-ad8b-fef030cbd02a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:43.463193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f10e0-10dc-4e93-ad8b-fef030cbd02a service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:43.476782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 706/1416] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:44.542540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067dbad-8941-434a-8e1d-d66cd0629178 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:44.543157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067dbad-8941-434a-8e1d-d66cd0629178 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:44.543157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067dbad-8941-434a-8e1d-d66cd0629178 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:44.544566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:44.549754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d6966-b5c5-455d-afa2-e3e57fceb95f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:44.550197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d6966-b5c5-455d-afa2-e3e57fceb95f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:44.550308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d6966-b5c5-455d-afa2-e3e57fceb95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:44.571445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-725d6966-b5c5-455d-afa2-e3e57fceb95f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:44.572662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-725d6966-b5c5-455d-afa2-e3e57fceb95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:44.580650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:44.741677 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067dbad-8941-434a-8e1d-d66cd0629178 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:44.756763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067dbad-8941-434a-8e1d-d66cd0629178 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:44.762134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70e874-8ed8-4a2a-bf62-999ae178e5fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:44.762793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70e874-8ed8-4a2a-bf62-999ae178e5fe service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:44.763072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70e874-8ed8-4a2a-bf62-999ae178e5fe service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:44.763344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70e874-8ed8-4a2a-bf62-999ae178e5fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:44.771280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70e874-8ed8-4a2a-bf62-999ae178e5fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:44.772837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70e874-8ed8-4a2a-bf62-999ae178e5fe service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:44.782143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 712/1419] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:45.773597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855ad06-b4eb-44cb-b208-897c841d660a None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:45.774144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855ad06-b4eb-44cb-b208-897c841d660a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:45.774144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855ad06-b4eb-44cb-b208-897c841d660a None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:45.775436 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:45.780570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4d440-4c43-4a0d-ba60-a185c94f9924 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:45.781049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4d440-4c43-4a0d-ba60-a185c94f9924 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:45.781157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4d440-4c43-4a0d-ba60-a185c94f9924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:45.799203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-76e4d440-4c43-4a0d-ba60-a185c94f9924 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:45.800424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-76e4d440-4c43-4a0d-ba60-a185c94f9924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:45.808863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:45.962502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855ad06-b4eb-44cb-b208-897c841d660a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:45.977642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855ad06-b4eb-44cb-b208-897c841d660a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:45.982839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0d234-33be-44f5-a2bd-6ab32d0a64fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:45.983478 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0d234-33be-44f5-a2bd-6ab32d0a64fe service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:45.983760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0d234-33be-44f5-a2bd-6ab32d0a64fe service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:45.984035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0d234-33be-44f5-a2bd-6ab32d0a64fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:45.992251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0d234-33be-44f5-a2bd-6ab32d0a64fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:45.993826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0d234-33be-44f5-a2bd-6ab32d0a64fe service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:46.002953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 709/1422] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:46.937588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c4e2b-0b5e-4822-92d9-2496f16e02ed None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:46.938174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c4e2b-0b5e-4822-92d9-2496f16e02ed None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:46.938174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c4e2b-0b5e-4822-92d9-2496f16e02ed None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:46.939557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:48:46.945205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4d2d9-6fb1-4700-b73d-ad8d09977ed5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:46.945703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4d2d9-6fb1-4700-b73d-ad8d09977ed5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:46.945813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4d2d9-6fb1-4700-b73d-ad8d09977ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:46.964690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-64e4d2d9-6fb1-4700-b73d-ad8d09977ed5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:46.965906 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-64e4d2d9-6fb1-4700-b73d-ad8d09977ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:46.973745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:47.123617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c4e2b-0b5e-4822-92d9-2496f16e02ed None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:47.138629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c4e2b-0b5e-4822-92d9-2496f16e02ed None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:47.144048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c681f5-cb7a-4061-92f4-6c05d1861f76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:47.144681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c681f5-cb7a-4061-92f4-6c05d1861f76 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:47.144965 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c681f5-cb7a-4061-92f4-6c05d1861f76 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:47.145238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c681f5-cb7a-4061-92f4-6c05d1861f76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:47.153655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c681f5-cb7a-4061-92f4-6c05d1861f76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:47.155266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c681f5-cb7a-4061-92f4-6c05d1861f76 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:47.165305 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 715/1425] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:48.133365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610cf10-f37b-481f-8398-fa71b81b59f5 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:48.134094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610cf10-f37b-481f-8398-fa71b81b59f5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:48.134094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610cf10-f37b-481f-8398-fa71b81b59f5 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:48.135245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:48:48.141391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75915e71-3662-4854-8d40-f62819bfd60c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:48.141949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75915e71-3662-4854-8d40-f62819bfd60c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:48.142094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75915e71-3662-4854-8d40-f62819bfd60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:48.161856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-75915e71-3662-4854-8d40-f62819bfd60c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:48.163703 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-75915e71-3662-4854-8d40-f62819bfd60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:48.175930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:48:48.342425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610cf10-f37b-481f-8398-fa71b81b59f5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:48.357520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610cf10-f37b-481f-8398-fa71b81b59f5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:48.362934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23badbb0-103b-41eb-a9d5-e86d654141d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:48.363568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23badbb0-103b-41eb-a9d5-e86d654141d5 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:48.363850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23badbb0-103b-41eb-a9d5-e86d654141d5 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:48.364123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23badbb0-103b-41eb-a9d5-e86d654141d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:48.372411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23badbb0-103b-41eb-a9d5-e86d654141d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:48.373983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23badbb0-103b-41eb-a9d5-e86d654141d5 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:48.383799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 712/1428] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:49.285409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b0ba0-a6c1-417c-966a-9bb660031ec2 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:49.286277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b0ba0-a6c1-417c-966a-9bb660031ec2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:49.286277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b0ba0-a6c1-417c-966a-9bb660031ec2 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:49.288032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:48:49.293638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149ea6e-5477-4d87-8464-b5587000c360 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:49.294136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149ea6e-5477-4d87-8464-b5587000c360 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:49.294283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149ea6e-5477-4d87-8464-b5587000c360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:49.315173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a149ea6e-5477-4d87-8464-b5587000c360 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:49.316388 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a149ea6e-5477-4d87-8464-b5587000c360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:49.325249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:49.507519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b0ba0-a6c1-417c-966a-9bb660031ec2 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:49.525600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b0ba0-a6c1-417c-966a-9bb660031ec2 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:49.531486 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a074a-da1d-4036-bd63-48b80e132470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:49.532132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a074a-da1d-4036-bd63-48b80e132470 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:49.533031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a074a-da1d-4036-bd63-48b80e132470 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:49.533031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a074a-da1d-4036-bd63-48b80e132470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:49.542196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a074a-da1d-4036-bd63-48b80e132470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:49.543929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a074a-da1d-4036-bd63-48b80e132470 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:49.554610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 718/1431] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:50.460045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ec3b6-c2bd-478d-b33e-6c78e251d25b None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:50.460614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ec3b6-c2bd-478d-b33e-6c78e251d25b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:50.460614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ec3b6-c2bd-478d-b33e-6c78e251d25b None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:50.461970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:50.467041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99dd16a-3540-4de9-b668-c91fd185172b None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:50.467494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99dd16a-3540-4de9-b668-c91fd185172b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:50.467594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99dd16a-3540-4de9-b668-c91fd185172b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:50.485245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b99dd16a-3540-4de9-b668-c91fd185172b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:50.486497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b99dd16a-3540-4de9-b668-c91fd185172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:50.495052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:50.647565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ec3b6-c2bd-478d-b33e-6c78e251d25b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:50.661468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ec3b6-c2bd-478d-b33e-6c78e251d25b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:50.668010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ededb-b236-4b5f-a0db-376dcf610025 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:50.668645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ededb-b236-4b5f-a0db-376dcf610025 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:50.668950 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ededb-b236-4b5f-a0db-376dcf610025 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:50.669225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ededb-b236-4b5f-a0db-376dcf610025 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:50.677588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ededb-b236-4b5f-a0db-376dcf610025 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:50.679886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ededb-b236-4b5f-a0db-376dcf610025 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:50.690236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 715/1434] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:51.576298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c63ed-fe38-4f69-a71e-efce0ad2c16b None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:51.576881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c63ed-fe38-4f69-a71e-efce0ad2c16b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:51.576881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c63ed-fe38-4f69-a71e-efce0ad2c16b None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:51.578222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:51.583259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fdd10-4b64-4175-98ff-8ef89f5feee8 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:51.583696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fdd10-4b64-4175-98ff-8ef89f5feee8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:51.583836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fdd10-4b64-4175-98ff-8ef89f5feee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:51.602328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-cb3fdd10-4b64-4175-98ff-8ef89f5feee8 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:51.603642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-cb3fdd10-4b64-4175-98ff-8ef89f5feee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:51.611485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:51.763261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c63ed-fe38-4f69-a71e-efce0ad2c16b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:51.779033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c63ed-fe38-4f69-a71e-efce0ad2c16b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:51.787795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566fc61-5659-48a9-88d3-299d8b9a5b77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:51.788583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566fc61-5659-48a9-88d3-299d8b9a5b77 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:51.789092 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566fc61-5659-48a9-88d3-299d8b9a5b77 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:51.789553 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566fc61-5659-48a9-88d3-299d8b9a5b77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:51.799784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566fc61-5659-48a9-88d3-299d8b9a5b77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:51.801390 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566fc61-5659-48a9-88d3-299d8b9a5b77 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:51.816122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 721/1437] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:52.744826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b59c0-d541-4e73-ac2b-5eaca28557cb None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:52.745435 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b59c0-d541-4e73-ac2b-5eaca28557cb None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:52.745555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b59c0-d541-4e73-ac2b-5eaca28557cb None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:52.747129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:48:52.753415 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f6282-623b-4621-a42e-fd5e9404ec33 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:52.754269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f6282-623b-4621-a42e-fd5e9404ec33 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:52.754432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f6282-623b-4621-a42e-fd5e9404ec33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:52.773931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f03f6282-623b-4621-a42e-fd5e9404ec33 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:52.775156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f03f6282-623b-4621-a42e-fd5e9404ec33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:52.784214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:48:52.934358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b59c0-d541-4e73-ac2b-5eaca28557cb None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:52.949600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b59c0-d541-4e73-ac2b-5eaca28557cb None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:52.955161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733defff-644a-4873-b43a-eb72e2e9aa4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:52.956032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733defff-644a-4873-b43a-eb72e2e9aa4f service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:52.956328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733defff-644a-4873-b43a-eb72e2e9aa4f service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:52.956596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733defff-644a-4873-b43a-eb72e2e9aa4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:52.966142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733defff-644a-4873-b43a-eb72e2e9aa4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:52.967775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733defff-644a-4873-b43a-eb72e2e9aa4f service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:52.978601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 718/1440] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:55.253254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b81df-4403-443e-8a5d-e4ad7c0aa16a None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:55.253798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b81df-4403-443e-8a5d-e4ad7c0aa16a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:55.253798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b81df-4403-443e-8a5d-e4ad7c0aa16a None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:55.255093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:55.261122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583892f8-97b3-4c05-bc77-f7b7988f2ffc None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:55.261580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583892f8-97b3-4c05-bc77-f7b7988f2ffc None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:55.261675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583892f8-97b3-4c05-bc77-f7b7988f2ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:55.280680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-583892f8-97b3-4c05-bc77-f7b7988f2ffc None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:55.281890 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-583892f8-97b3-4c05-bc77-f7b7988f2ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:55.289893 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:48:55.328804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b81df-4403-443e-8a5d-e4ad7c0aa16a None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:55.340894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b81df-4403-443e-8a5d-e4ad7c0aa16a None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:55.344472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f84a29-4116-4f69-ad65-9ae429b5edd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:55.345148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84a29-4116-4f69-ad65-9ae429b5edd1 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:55.345429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84a29-4116-4f69-ad65-9ae429b5edd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:55.345705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84a29-4116-4f69-ad65-9ae429b5edd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:55.355371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f84a29-4116-4f69-ad65-9ae429b5edd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:55.356825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f84a29-4116-4f69-ad65-9ae429b5edd1 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:55.367375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 724/1443] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:48:57.830384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906b8de-555c-45d3-8fdc-89edf18257aa None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:57.830967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906b8de-555c-45d3-8fdc-89edf18257aa None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:57.831147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906b8de-555c-45d3-8fdc-89edf18257aa None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:57.832728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:48:57.838491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e44f4f-2018-45bc-a0a5-d5af0a0d925e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:57.839052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e44f4f-2018-45bc-a0a5-d5af0a0d925e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:57.839052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e44f4f-2018-45bc-a0a5-d5af0a0d925e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:57.856511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-32e44f4f-2018-45bc-a0a5-d5af0a0d925e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:48:57.857707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-32e44f4f-2018-45bc-a0a5-d5af0a0d925e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:48:57.866058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:48:57.904114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906b8de-555c-45d3-8fdc-89edf18257aa None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:48:57.915169 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906b8de-555c-45d3-8fdc-89edf18257aa None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:48:57.918839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeeb90-44dd-43ff-86d1-77634a16bb25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:48:57.919477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeeb90-44dd-43ff-86d1-77634a16bb25 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:48:57.919856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeeb90-44dd-43ff-86d1-77634a16bb25 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:48:57.920025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeeb90-44dd-43ff-86d1-77634a16bb25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:48:57.930159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeeb90-44dd-43ff-86d1-77634a16bb25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:48:57.931649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeeb90-44dd-43ff-86d1-77634a16bb25 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:48:57.941539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 721/1446] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:48:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:49:00.612474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203d9e1-b856-4693-8980-43cec5c6d90b None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:00.613049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203d9e1-b856-4693-8980-43cec5c6d90b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:00.613049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203d9e1-b856-4693-8980-43cec5c6d90b None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:00.614307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:49:00.620165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cb185-f796-4f38-9a9a-f3b7c6918889 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:00.620658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cb185-f796-4f38-9a9a-f3b7c6918889 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:00.620862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cb185-f796-4f38-9a9a-f3b7c6918889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:00.638927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-561cb185-f796-4f38-9a9a-f3b7c6918889 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:00.640129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-561cb185-f796-4f38-9a9a-f3b7c6918889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:00.649018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:49:00.686962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203d9e1-b856-4693-8980-43cec5c6d90b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:00.698648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203d9e1-b856-4693-8980-43cec5c6d90b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:00.702294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b415c5c-3cf5-411c-b04d-80a6b16ac9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:00.702963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b415c5c-3cf5-411c-b04d-80a6b16ac9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:00.703247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b415c5c-3cf5-411c-b04d-80a6b16ac9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:00.703518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b415c5c-3cf5-411c-b04d-80a6b16ac9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:00.713976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b415c5c-3cf5-411c-b04d-80a6b16ac9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:00.715441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b415c5c-3cf5-411c-b04d-80a6b16ac9d3 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:00.726432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 727/1449] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:49:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:49:03.139850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec640-8f5e-4d83-b994-dffd8f57b820 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:03.140474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec640-8f5e-4d83-b994-dffd8f57b820 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:03.140474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec640-8f5e-4d83-b994-dffd8f57b820 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:03.141779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:49:03.147881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da2421-63d5-4905-9f27-03da47872bf0 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:03.148392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da2421-63d5-4905-9f27-03da47872bf0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:03.148492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da2421-63d5-4905-9f27-03da47872bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:03.169945 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-67da2421-63d5-4905-9f27-03da47872bf0 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:03.171575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-67da2421-63d5-4905-9f27-03da47872bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:03.183670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:49:03.227202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec640-8f5e-4d83-b994-dffd8f57b820 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:03.239912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec640-8f5e-4d83-b994-dffd8f57b820 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:03.244149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8186c38-1e0d-40dc-993e-01fa16703212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:03.244804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8186c38-1e0d-40dc-993e-01fa16703212 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:03.245079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8186c38-1e0d-40dc-993e-01fa16703212 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:03.245348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8186c38-1e0d-40dc-993e-01fa16703212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:03.264853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8186c38-1e0d-40dc-993e-01fa16703212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:03.266283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8186c38-1e0d-40dc-993e-01fa16703212 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:03.277346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 724/1452] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:49:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:49:05.829351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c2d86-1290-48c4-bb9c-c70a86975232 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:05.830481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c2d86-1290-48c4-bb9c-c70a86975232 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:05.830481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c2d86-1290-48c4-bb9c-c70a86975232 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:05.831367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:49:05.839299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781e00-237c-4aee-84eb-73e28eb6652d None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:05.840163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781e00-237c-4aee-84eb-73e28eb6652d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:05.840564 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781e00-237c-4aee-84eb-73e28eb6652d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:05.859624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7d781e00-237c-4aee-84eb-73e28eb6652d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:05.860814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7d781e00-237c-4aee-84eb-73e28eb6652d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:05.868623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:49:05.914880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c2d86-1290-48c4-bb9c-c70a86975232 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:05.926468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c2d86-1290-48c4-bb9c-c70a86975232 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:05.930544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95765f95-78f2-484c-9f63-df127f9ed8b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:05.931235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95765f95-78f2-484c-9f63-df127f9ed8b3 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:05.931516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95765f95-78f2-484c-9f63-df127f9ed8b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:05.931795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95765f95-78f2-484c-9f63-df127f9ed8b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:05.945740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95765f95-78f2-484c-9f63-df127f9ed8b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:05.947841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95765f95-78f2-484c-9f63-df127f9ed8b3 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:05.961819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 730/1455] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:49:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:49:08.325408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbdea6-e856-45e1-a7c0-efb2705655ef None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:08.325898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbdea6-e856-45e1-a7c0-efb2705655ef None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:08.326014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbdea6-e856-45e1-a7c0-efb2705655ef None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:08.327373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:49:08.333116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003690be-d8ca-4895-81ca-3e80e67bca16 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:08.333594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003690be-d8ca-4895-81ca-3e80e67bca16 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:08.333734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003690be-d8ca-4895-81ca-3e80e67bca16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:08.353225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-003690be-d8ca-4895-81ca-3e80e67bca16 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:08.354435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-003690be-d8ca-4895-81ca-3e80e67bca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:08.362497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:49:08.402666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbdea6-e856-45e1-a7c0-efb2705655ef None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:08.414159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbdea6-e856-45e1-a7c0-efb2705655ef None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:08.417753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d523e9-a714-4849-ac32-bf8e4a9260fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:08.418397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d523e9-a714-4849-ac32-bf8e4a9260fa service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:08.418684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d523e9-a714-4849-ac32-bf8e4a9260fa service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:08.418998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d523e9-a714-4849-ac32-bf8e4a9260fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:08.428707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d523e9-a714-4849-ac32-bf8e4a9260fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:08.430188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d523e9-a714-4849-ac32-bf8e4a9260fa service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:08.440043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 727/1458] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:49:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:49:10.735579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4b2ef-5f42-43f4-b47d-f36d2e5b8047 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:10.736188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4b2ef-5f42-43f4-b47d-f36d2e5b8047 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:10.736188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4b2ef-5f42-43f4-b47d-f36d2e5b8047 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:10.737475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:49:10.743144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b624c-ce51-4469-bc99-fd4ac46f2e9c None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:10.743641 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b624c-ce51-4469-bc99-fd4ac46f2e9c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:10.744205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b624c-ce51-4469-bc99-fd4ac46f2e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:10.765330 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e17b624c-ce51-4469-bc99-fd4ac46f2e9c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:10.766542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e17b624c-ce51-4469-bc99-fd4ac46f2e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:10.776285 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:49:10.813223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4b2ef-5f42-43f4-b47d-f36d2e5b8047 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:10.824557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4b2ef-5f42-43f4-b47d-f36d2e5b8047 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:10.828177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164c566-0bcc-4702-8e2f-93feb6de7a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:10.828814 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164c566-0bcc-4702-8e2f-93feb6de7a4b service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:10.829099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164c566-0bcc-4702-8e2f-93feb6de7a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:10.829368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164c566-0bcc-4702-8e2f-93feb6de7a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:10.838951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164c566-0bcc-4702-8e2f-93feb6de7a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:10.840372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164c566-0bcc-4702-8e2f-93feb6de7a4b service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:10.850152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 733/1461] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:49:13.219845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687177-008f-4b1b-b05b-fd540b8f8126 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:13.220575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687177-008f-4b1b-b05b-fd540b8f8126 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:13.220575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687177-008f-4b1b-b05b-fd540b8f8126 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:13.221765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:49:13.228432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b826b-9e8c-44af-88ec-76202df00de2 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:13.229018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b826b-9e8c-44af-88ec-76202df00de2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:13.229018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b826b-9e8c-44af-88ec-76202df00de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:13.251094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-028b826b-9e8c-44af-88ec-76202df00de2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:13.252446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-028b826b-9e8c-44af-88ec-76202df00de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:13.260897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:49:13.301238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59687177-008f-4b1b-b05b-fd540b8f8126 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:13.314541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59687177-008f-4b1b-b05b-fd540b8f8126 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:13.318757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4a86f-4377-423f-a325-cf088d146142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:13.319416 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4a86f-4377-423f-a325-cf088d146142 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:13.319705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4a86f-4377-423f-a325-cf088d146142 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:13.319984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4a86f-4377-423f-a325-cf088d146142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:13.330615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4a86f-4377-423f-a325-cf088d146142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:13.332132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4a86f-4377-423f-a325-cf088d146142 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:13.343570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 730/1464] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:49:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:49:15.719808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823ede0-7c62-4175-a473-6c57a33468db None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:15.720366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823ede0-7c62-4175-a473-6c57a33468db None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:15.720656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823ede0-7c62-4175-a473-6c57a33468db None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:15.723424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:49:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:49:15.732986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7894b-836c-431b-be4a-8cc79b502151 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:15.733765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7894b-836c-431b-be4a-8cc79b502151 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:15.733765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7894b-836c-431b-be4a-8cc79b502151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:15.754037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-14e7894b-836c-431b-be4a-8cc79b502151 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:15.755328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-14e7894b-836c-431b-be4a-8cc79b502151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:15.764131 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:49:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:49:15.800169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823ede0-7c62-4175-a473-6c57a33468db None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:15.812569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823ede0-7c62-4175-a473-6c57a33468db None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:15.817007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830887b6-290c-4233-bb3c-4be6fe9b4a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:15.817785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830887b6-290c-4233-bb3c-4be6fe9b4a3a service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:15.819086 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830887b6-290c-4233-bb3c-4be6fe9b4a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:15.819411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830887b6-290c-4233-bb3c-4be6fe9b4a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:15.829959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830887b6-290c-4233-bb3c-4be6fe9b4a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:15.832154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830887b6-290c-4233-bb3c-4be6fe9b4a3a service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:15.843476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 736/1467] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:49:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:49:18.184500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555bd6-3730-4cf0-b6e1-a31798715473 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:18.185065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555bd6-3730-4cf0-b6e1-a31798715473 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:18.185065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555bd6-3730-4cf0-b6e1-a31798715473 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:18.186614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:49:18.193908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479be318-9331-4c95-994f-c2a73f18c821 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:18.194743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479be318-9331-4c95-994f-c2a73f18c821 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:18.194743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479be318-9331-4c95-994f-c2a73f18c821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:18.219171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-479be318-9331-4c95-994f-c2a73f18c821 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:18.221006 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-479be318-9331-4c95-994f-c2a73f18c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:18.233223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:49:18.270842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48555bd6-3730-4cf0-b6e1-a31798715473 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:18.281656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48555bd6-3730-4cf0-b6e1-a31798715473 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:18.285350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ca5a9-b64a-4483-9603-193282fde50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:18.286001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ca5a9-b64a-4483-9603-193282fde50f service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:18.286282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ca5a9-b64a-4483-9603-193282fde50f service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:18.286558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ca5a9-b64a-4483-9603-193282fde50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:18.296406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ca5a9-b64a-4483-9603-193282fde50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:18.297846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ca5a9-b64a-4483-9603-193282fde50f service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:18.308055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 733/1470] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:49:52.666609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d23aca-b6b5-444b-a45b-58e310670a14 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:52.667215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d23aca-b6b5-444b-a45b-58e310670a14 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:52.667215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d23aca-b6b5-444b-a45b-58e310670a14 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:52.668488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 738/1471] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:49:52.673874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd04490-433a-4570-a591-2deb9b6c4bd5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:52.674312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd04490-433a-4570-a591-2deb9b6c4bd5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:52.674420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd04490-433a-4570-a591-2deb9b6c4bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:52.692214 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1cd04490-433a-4570-a591-2deb9b6c4bd5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:52.694154 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1cd04490-433a-4570-a591-2deb9b6c4bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:52.702162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 734/1472] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:49:52.748485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee697e6-7c59-41ba-b44b-50f4dec497a3 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:52.748771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee697e6-7c59-41ba-b44b-50f4dec497a3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:52.749074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee697e6-7c59-41ba-b44b-50f4dec497a3 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:52.750456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 739/1473] 10.222.0.79 () {54 vars in 962 bytes} [Fri May 13 13:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:49:52.756701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f18f3-567b-4148-b720-89355cf14c52 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:52.757020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f18f3-567b-4148-b720-89355cf14c52 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:52.757325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f18f3-567b-4148-b720-89355cf14c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:52.774863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-647f18f3-567b-4148-b720-89355cf14c52 None None] MFA Rules not processed for user `cc4f45e859d84ceaa5a01477e83e098e`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:52.776054 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-647f18f3-567b-4148-b720-89355cf14c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:52.782442 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 735/1474] 10.222.0.79 () {58 vars in 1059 bytes} [Fri May 13 13:49:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:49:52.789711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee697e6-7c59-41ba-b44b-50f4dec497a3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:52.800473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee697e6-7c59-41ba-b44b-50f4dec497a3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:52.804572 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a66925-3bcd-4f5f-b578-210079da62c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4f45e859d84ceaa5a01477e83e098e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:52.805210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a66925-3bcd-4f5f-b578-210079da62c0 service glance] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:52.805496 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a66925-3bcd-4f5f-b578-210079da62c0 service glance] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:52.805769 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a66925-3bcd-4f5f-b578-210079da62c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:52.814809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a66925-3bcd-4f5f-b578-210079da62c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:52.816232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a66925-3bcd-4f5f-b578-210079da62c0 service glance] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:52.826437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 740/1475] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:49:55.191859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a67b9f-1a51-4d70-8b74-97349ebf600e None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:55.193221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a67b9f-1a51-4d70-8b74-97349ebf600e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:55.193221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a67b9f-1a51-4d70-8b74-97349ebf600e None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:55.194628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 736/1476] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:49:55.201576 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3c552-06ad-4398-b607-715a35f8308e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:55.202067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3c552-06ad-4398-b607-715a35f8308e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:55.202228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3c552-06ad-4398-b607-715a35f8308e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:55.222415 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-77a3c552-06ad-4398-b607-715a35f8308e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:55.223619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-77a3c552-06ad-4398-b607-715a35f8308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:55.231943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 741/1477] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:49:55.256579 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a67b9f-1a51-4d70-8b74-97349ebf600e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:55.268610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a67b9f-1a51-4d70-8b74-97349ebf600e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:55.273776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c9dd0-d9b5-45b2-9940-20ae4f877405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:55.274438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9dd0-d9b5-45b2-9940-20ae4f877405 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:55.274745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9dd0-d9b5-45b2-9940-20ae4f877405 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:55.275024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9dd0-d9b5-45b2-9940-20ae4f877405 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:55.275759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c9dd0-d9b5-45b2-9940-20ae4f877405 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:55.277559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WARNING py.warnings [None req-540c9dd0-d9b5-45b2-9940-20ae4f877405 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:49:55.277559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: warnings.warn(msg) May 13 13:49:55.277559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: May 13 13:49:55.278699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c9dd0-d9b5-45b2-9940-20ae4f877405 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:55.298934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 737/1478] 10.222.0.79 () {56 vars in 1108 bytes} [Fri May 13 13:49:55 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:49:57.698493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dea638-cf18-4b6c-a791-a1cf21e0ef67 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:57.699112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dea638-cf18-4b6c-a791-a1cf21e0ef67 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:57.699112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dea638-cf18-4b6c-a791-a1cf21e0ef67 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:57.700498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 742/1479] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:49:57.706591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89736-e942-4d5c-a376-4ecc18c37e93 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:57.707254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89736-e942-4d5c-a376-4ecc18c37e93 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:57.707254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89736-e942-4d5c-a376-4ecc18c37e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:57.730357 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e0e89736-e942-4d5c-a376-4ecc18c37e93 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:49:57.731649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e0e89736-e942-4d5c-a376-4ecc18c37e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:49:57.740752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 738/1480] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:49:57.789068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dea638-cf18-4b6c-a791-a1cf21e0ef67 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:49:57.803255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dea638-cf18-4b6c-a791-a1cf21e0ef67 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:49:57.808542 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6b675-e36b-4bc4-8217-4ba1e58ef0ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:49:57.809164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6b675-e36b-4bc4-8217-4ba1e58ef0ab service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:49:57.809439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6b675-e36b-4bc4-8217-4ba1e58ef0ab service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:49:57.809708 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6b675-e36b-4bc4-8217-4ba1e58ef0ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:49:57.821945 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6b675-e36b-4bc4-8217-4ba1e58ef0ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:49:57.823574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6b675-e36b-4bc4-8217-4ba1e58ef0ab service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:49:57.834100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 743/1481] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:50:00.481320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb60f91-f70d-4f09-ad5d-f3a4a6d43ddf None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:00.481888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb60f91-f70d-4f09-ad5d-f3a4a6d43ddf None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:00.481888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb60f91-f70d-4f09-ad5d-f3a4a6d43ddf None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:00.483308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 739/1482] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:50:00.489418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e657ec8-fb57-460d-bcb8-18f4be826aa6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:00.489908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e657ec8-fb57-460d-bcb8-18f4be826aa6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:00.490156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e657ec8-fb57-460d-bcb8-18f4be826aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:00.508054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2e657ec8-fb57-460d-bcb8-18f4be826aa6 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:50:00.508919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2e657ec8-fb57-460d-bcb8-18f4be826aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:50:00.516884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 744/1483] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:50:00.553406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb60f91-f70d-4f09-ad5d-f3a4a6d43ddf None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:50:00.564116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb60f91-f70d-4f09-ad5d-f3a4a6d43ddf None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:50:00.568073 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa89b2-45c3-4d17-a37d-33c68794f923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:50:00.568711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa89b2-45c3-4d17-a37d-33c68794f923 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:00.568987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa89b2-45c3-4d17-a37d-33c68794f923 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:00.569265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa89b2-45c3-4d17-a37d-33c68794f923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:00.578205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa89b2-45c3-4d17-a37d-33c68794f923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:50:00.579717 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa89b2-45c3-4d17-a37d-33c68794f923 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:50:00.589306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 740/1484] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:50:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:50:02.917545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a464f42-0657-4a77-8e87-a549cd854a23 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:02.917545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a464f42-0657-4a77-8e87-a549cd854a23 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:02.918249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a464f42-0657-4a77-8e87-a549cd854a23 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:02.920991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 745/1485] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:50:02.925690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5e13d-2a75-4ab6-ba18-7b206eb972b6 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:02.926376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5e13d-2a75-4ab6-ba18-7b206eb972b6 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:02.926376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5e13d-2a75-4ab6-ba18-7b206eb972b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:02.944617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-5de5e13d-2a75-4ab6-ba18-7b206eb972b6 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:50:02.945838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5de5e13d-2a75-4ab6-ba18-7b206eb972b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:50:02.953742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 741/1486] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:50:02.994008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a464f42-0657-4a77-8e87-a549cd854a23 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:50:03.004968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a464f42-0657-4a77-8e87-a549cd854a23 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:50:03.008488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea75997-3189-466f-9c8a-06e3353c190b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:50:03.009153 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea75997-3189-466f-9c8a-06e3353c190b service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:03.009436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea75997-3189-466f-9c8a-06e3353c190b service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:03.009733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea75997-3189-466f-9c8a-06e3353c190b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:03.019003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea75997-3189-466f-9c8a-06e3353c190b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:50:03.020467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea75997-3189-466f-9c8a-06e3353c190b service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:50:03.030049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 746/1487] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:50:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:50:05.392552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b05998-a8c1-4f28-a271-88b21bb96cb2 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:05.393069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b05998-a8c1-4f28-a271-88b21bb96cb2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:05.393185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b05998-a8c1-4f28-a271-88b21bb96cb2 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:05.394650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 742/1488] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:50:05.400807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15307c83-7e39-43b9-addd-c535e0634e21 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:05.401561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15307c83-7e39-43b9-addd-c535e0634e21 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:05.401561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15307c83-7e39-43b9-addd-c535e0634e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:05.420138 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-15307c83-7e39-43b9-addd-c535e0634e21 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:50:05.421329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-15307c83-7e39-43b9-addd-c535e0634e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:50:05.429255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 747/1489] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:50:05.470133 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b05998-a8c1-4f28-a271-88b21bb96cb2 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:50:05.481898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b05998-a8c1-4f28-a271-88b21bb96cb2 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:50:05.485512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b4993-732c-413a-ac9d-9e1a24c7bffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:50:05.486165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b4993-732c-413a-ac9d-9e1a24c7bffa service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:05.486443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b4993-732c-413a-ac9d-9e1a24c7bffa service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:05.486739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b4993-732c-413a-ac9d-9e1a24c7bffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:05.505064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b4993-732c-413a-ac9d-9e1a24c7bffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:50:05.506554 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b4993-732c-413a-ac9d-9e1a24c7bffa service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:50:05.517272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 743/1490] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:50:08.263615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fee6f-a65a-464b-8865-e7f420a6f489 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:08.264389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fee6f-a65a-464b-8865-e7f420a6f489 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:08.264389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fee6f-a65a-464b-8865-e7f420a6f489 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:08.265536 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 748/1491] 10.222.0.79 () {54 vars in 926 bytes} [Fri May 13 13:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:50:08.271383 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369dd54-c29e-4ac1-ba27-161f3e81a59a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:08.271950 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369dd54-c29e-4ac1-ba27-161f3e81a59a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:08.271950 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369dd54-c29e-4ac1-ba27-161f3e81a59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:08.289675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-0369dd54-c29e-4ac1-ba27-161f3e81a59a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:50:08.290923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0369dd54-c29e-4ac1-ba27-161f3e81a59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:50:08.298884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 744/1492] 10.222.0.79 () {58 vars in 1023 bytes} [Fri May 13 13:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:50:08.340139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fee6f-a65a-464b-8865-e7f420a6f489 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:50:08.351385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fee6f-a65a-464b-8865-e7f420a6f489 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:50:08.355045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111041f-9d73-4b5d-82f4-c6a80b3728d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:50:08.355778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111041f-9d73-4b5d-82f4-c6a80b3728d4 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:50:08.355960 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111041f-9d73-4b5d-82f4-c6a80b3728d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:50:08.356240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111041f-9d73-4b5d-82f4-c6a80b3728d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:50:08.367074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111041f-9d73-4b5d-82f4-c6a80b3728d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:50:08.368530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111041f-9d73-4b5d-82f4-c6a80b3728d4 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:50:08.378841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 749/1493] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:51:47.896497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5947d-a514-44d3-9b00-2bb664a09b08 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:47.897328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5947d-a514-44d3-9b00-2bb664a09b08 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:47.897328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5947d-a514-44d3-9b00-2bb664a09b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:47.926359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-54f5947d-a514-44d3-9b00-2bb664a09b08 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:51:47.928580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-54f5947d-a514-44d3-9b00-2bb664a09b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:51:47.943573 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 745/1494] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:51:47.955634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6123db-bdab-48da-ac89-a218b202f832 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:47.956238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6123db-bdab-48da-ac89-a218b202f832 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:47.956376 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6123db-bdab-48da-ac89-a218b202f832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:47.975947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6b6123db-bdab-48da-ac89-a218b202f832 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:51:47.977189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6b6123db-bdab-48da-ac89-a218b202f832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:51:47.985904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 750/1495] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:51:47.992368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5947d-a514-44d3-9b00-2bb664a09b08 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:51:48.009903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5947d-a514-44d3-9b00-2bb664a09b08 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:51:48.018341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bac64-d9da-45ca-8cce-d4474c36b3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:51:48.019540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bac64-d9da-45ca-8cce-d4474c36b3ac admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.020032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bac64-d9da-45ca-8cce-d4474c36b3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.020523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bac64-d9da-45ca-8cce-d4474c36b3ac admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.021924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bac64-d9da-45ca-8cce-d4474c36b3ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:51:48.022462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bac64-d9da-45ca-8cce-d4474c36b3ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:51:48.024654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WARNING py.warnings [None req-6e1bac64-d9da-45ca-8cce-d4474c36b3ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 13:51:48.024654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: warnings.warn(msg) May 13 13:51:48.024654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: May 13 13:51:48.026343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bac64-d9da-45ca-8cce-d4474c36b3ac admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:51:48.048545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 746/1496] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:51:48.057471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6123db-bdab-48da-ac89-a218b202f832 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:51:48.067804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6123db-bdab-48da-ac89-a218b202f832 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:51:48.072862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50c189-1d63-4b3e-b721-843a6576c8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:51:48.073698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50c189-1d63-4b3e-b721-843a6576c8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.074022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50c189-1d63-4b3e-b721-843a6576c8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.074380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50c189-1d63-4b3e-b721-843a6576c8a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.075262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50c189-1d63-4b3e-b721-843a6576c8a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:51:48.077451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: WARNING py.warnings [None req-ed50c189-1d63-4b3e-b721-843a6576c8a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:51:48.077451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: warnings.warn(msg) May 13 13:51:48.077451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: May 13 13:51:48.078725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50c189-1d63-4b3e-b721-843a6576c8a8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:51:48.130926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 751/1497] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:51:48 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:51:48.136981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bac64-d9da-45ca-8cce-d4474c36b3ac admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:51:48.148204 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bac64-d9da-45ca-8cce-d4474c36b3ac admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:51:48.221439 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b26fbb-284a-4551-8175-b7118d968055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:51:48.222291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b26fbb-284a-4551-8175-b7118d968055 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.222585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b26fbb-284a-4551-8175-b7118d968055 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.222897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b26fbb-284a-4551-8175-b7118d968055 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.223601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b26fbb-284a-4551-8175-b7118d968055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:51:48.224647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WARNING py.warnings [None req-59b26fbb-284a-4551-8175-b7118d968055 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:51:48.224647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: warnings.warn(msg) May 13 13:51:48.224647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: May 13 13:51:48.225929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b26fbb-284a-4551-8175-b7118d968055 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:51:48.249979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 747/1498] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:51:48 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:51:48.255327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50c189-1d63-4b3e-b721-843a6576c8a8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:51:48.266241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50c189-1d63-4b3e-b721-843a6576c8a8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:51:48.271552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b0ef0-122a-435d-9cfd-a7266e03df8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:51:48.272266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b0ef0-122a-435d-9cfd-a7266e03df8c admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.272583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b0ef0-122a-435d-9cfd-a7266e03df8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.272931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b0ef0-122a-435d-9cfd-a7266e03df8c admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.273697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b0ef0-122a-435d-9cfd-a7266e03df8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:51:48.274992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: WARNING py.warnings [None req-eb2b0ef0-122a-435d-9cfd-a7266e03df8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 13:51:48.274992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: warnings.warn(msg) May 13 13:51:48.274992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: May 13 13:51:48.275943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b0ef0-122a-435d-9cfd-a7266e03df8c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:51:48.290974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 752/1499] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:51:48.296099 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b26fbb-284a-4551-8175-b7118d968055 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:51:48.315969 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b26fbb-284a-4551-8175-b7118d968055 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:51:48.324147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e764bc-3664-421a-bb15-8c81396ee171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:51:48.325321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e764bc-3664-421a-bb15-8c81396ee171 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.325677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e764bc-3664-421a-bb15-8c81396ee171 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.326179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e764bc-3664-421a-bb15-8c81396ee171 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.327654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e764bc-3664-421a-bb15-8c81396ee171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:51:48.329604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WARNING py.warnings [None req-c1e764bc-3664-421a-bb15-8c81396ee171 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 13:51:48.329604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: warnings.warn(msg) May 13 13:51:48.329604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: May 13 13:51:48.331953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e764bc-3664-421a-bb15-8c81396ee171 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:51:48.359885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 748/1500] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:51:48.364147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b0ef0-122a-435d-9cfd-a7266e03df8c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:51:48.375280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b0ef0-122a-435d-9cfd-a7266e03df8c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:51:48.381749 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886921b-0d1e-49e2-bdca-1c8db0b57a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:51:48.382816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886921b-0d1e-49e2-bdca-1c8db0b57a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.383133 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886921b-0d1e-49e2-bdca-1c8db0b57a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.383455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886921b-0d1e-49e2-bdca-1c8db0b57a13 admin admin] PATH_INFO: `/v3/projects/ce041aca2ed34785af6048fde96b945b/users/43e941fd8b2243cea69fa5f5128d8cd0/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.401060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886921b-0d1e-49e2-bdca-1c8db0b57a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce041aca2ed34785af6048fde96b945b, user_id=43e941fd8b2243cea69fa5f5128d8cd0, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:51:48.402273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: WARNING py.warnings [None req-0886921b-0d1e-49e2-bdca-1c8db0b57a13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:51:48.402273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: warnings.warn(msg) May 13 13:51:48.402273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: May 13 13:51:48.404561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886921b-0d1e-49e2-bdca-1c8db0b57a13 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:51:48.422943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 753/1501] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:51:48 2022] PUT /identity/v3/projects/ce041aca2ed34785af6048fde96b945b/users/43e941fd8b2243cea69fa5f5128d8cd0/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:51:48.436123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e7625-6fa8-4324-acd7-60428d826f64 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.436479 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e7625-6fa8-4324-acd7-60428d826f64 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.436772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e7625-6fa8-4324-acd7-60428d826f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.466854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-941e7625-6fa8-4324-acd7-60428d826f64 None None] MFA Rules not processed for user `43e941fd8b2243cea69fa5f5128d8cd0`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:51:48.468092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-941e7625-6fa8-4324-acd7-60428d826f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:51:48.543037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 749/1502] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:51:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:51:48.563286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b091abc-7028-4a78-ad25-6fd5edf3eea6 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.563602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b091abc-7028-4a78-ad25-6fd5edf3eea6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.563864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b091abc-7028-4a78-ad25-6fd5edf3eea6 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.566127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 754/1503] 10.222.0.79 () {54 vars in 963 bytes} [Fri May 13 13:51:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:51:48.572162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82f4f7-73f6-4450-8026-1f5bba945a3d None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.572454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82f4f7-73f6-4450-8026-1f5bba945a3d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.572736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82f4f7-73f6-4450-8026-1f5bba945a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.593056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-8b82f4f7-73f6-4450-8026-1f5bba945a3d None None] MFA Rules not processed for user `e03fa33a68ff4567bc25c9611cd9ae8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:51:48.594315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8b82f4f7-73f6-4450-8026-1f5bba945a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:51:48.649277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 750/1504] 10.222.0.79 () {58 vars in 1060 bytes} [Fri May 13 13:51:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:51:48.657626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b091abc-7028-4a78-ad25-6fd5edf3eea6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:51:48.668797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b091abc-7028-4a78-ad25-6fd5edf3eea6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:51:48.673037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7bd3e-9fed-4429-a991-49a8e9c108ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:51:48.674872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7bd3e-9fed-4429-a991-49a8e9c108ea service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.677408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7bd3e-9fed-4429-a991-49a8e9c108ea service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.677707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7bd3e-9fed-4429-a991-49a8e9c108ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.694022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7bd3e-9fed-4429-a991-49a8e9c108ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:51:48.695502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7bd3e-9fed-4429-a991-49a8e9c108ea service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:51:48.706552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 755/1505] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:51:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:51:48.753331 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342f50-e66c-4ff2-a035-9fca1eff46f8 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.753616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342f50-e66c-4ff2-a035-9fca1eff46f8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.753902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342f50-e66c-4ff2-a035-9fca1eff46f8 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.755309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 751/1506] 10.222.0.79 () {52 vars in 829 bytes} [Fri May 13 13:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:51:48.759547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7bd3e-9fed-4429-a991-49a8e9c108ea service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:51:48.836845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7bd3e-9fed-4429-a991-49a8e9c108ea service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:51:48.842129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:51:48.843089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.843366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.843636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] PATH_INFO: `/v3/users/43e941fd8b2243cea69fa5f5128d8cd0` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.844883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e941fd8b2243cea69fa5f5128d8cd0)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:51:48.846503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: WARNING py.warnings [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:51:48.846503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: warnings.warn(msg) May 13 13:51:48.846503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: May 13 13:51:48.847776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:51:48.921443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e941fd8b2243cea69fa5f5128d8cd0'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:51:48.945019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e941fd8b2243cea69fa5f5128d8cd0'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:51:48.959389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e941fd8b2243cea69fa5f5128d8cd0'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:51:48.960249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e941fd8b2243cea69fa5f5128d8cd0'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:51:48.969680 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e941fd8b2243cea69fa5f5128d8cd0'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:51:48.977020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 756/1507] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:51:48 2022] DELETE /identity/v3/users/43e941fd8b2243cea69fa5f5128d8cd0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:51:48.981580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7c933-2274-4401-869f-7aa5b48b8261 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:48.982044 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7c933-2274-4401-869f-7aa5b48b8261 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:48.982170 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7c933-2274-4401-869f-7aa5b48b8261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:48.999905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c4f7c933-2274-4401-869f-7aa5b48b8261 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:51:49.001099 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c4f7c933-2274-4401-869f-7aa5b48b8261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:51:49.061400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 752/1508] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:51:49.071769 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:51:49.073175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:51:49.131017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf01445-17f3-4c9f-8b7e-0652593b9a33 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:51:49.134446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207518e-2a59-4529-8b4f-c638acd36e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:51:49.135118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207518e-2a59-4529-8b4f-c638acd36e50 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:49.135399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207518e-2a59-4529-8b4f-c638acd36e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:49.135673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207518e-2a59-4529-8b4f-c638acd36e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:49.144689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207518e-2a59-4529-8b4f-c638acd36e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:51:49.146131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207518e-2a59-4529-8b4f-c638acd36e50 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:51:49.156495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 757/1509] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:51:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:51:49.553959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7c933-2274-4401-869f-7aa5b48b8261 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:51:49.556420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c4f7c933-2274-4401-869f-7aa5b48b8261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:51:49.577757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7c933-2274-4401-869f-7aa5b48b8261 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:51:49.583280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:51:49.584216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:51:49.584740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:51:49.585057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] PATH_INFO: `/v3/projects/ce041aca2ed34785af6048fde96b945b` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:51:49.586364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce041aca2ed34785af6048fde96b945b)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:51:49.587423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WARNING py.warnings [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:51:49.587423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: warnings.warn(msg) May 13 13:51:49.587423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: May 13 13:51:49.588633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:51:49.656464 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce041aca2ed34785af6048fde96b945b', 'request_id': 'req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:51:49.670526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce041aca2ed34785af6048fde96b945b', 'request_id': 'req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:51:49.676579 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce041aca2ed34785af6048fde96b945b', 'request_id': 'req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:51:49.691929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] The token cache is being invalidate because project ce041aca2ed34785af6048fde96b945b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:51:49.692207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:51:49.692911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b34adfb-37fd-4a4b-b5d2-19ddec5fdb9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:51:49.694596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 753/1510] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:51:49 2022] DELETE /identity/v3/projects/ce041aca2ed34785af6048fde96b945b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:53:36.480742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d7084-623d-42e4-913d-091b57f5bfeb None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:36.481338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d7084-623d-42e4-913d-091b57f5bfeb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:36.481515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d7084-623d-42e4-913d-091b57f5bfeb None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:36.483247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:53:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:53:36.489518 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c840e7f-17e5-4be8-a51e-dffafd04885f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:36.490258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c840e7f-17e5-4be8-a51e-dffafd04885f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:36.490258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c840e7f-17e5-4be8-a51e-dffafd04885f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:36.510141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1c840e7f-17e5-4be8-a51e-dffafd04885f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:53:36.511424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1c840e7f-17e5-4be8-a51e-dffafd04885f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:36.578164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:53:36.756483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d7084-623d-42e4-913d-091b57f5bfeb None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:53:36.758352 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d17d7084-623d-42e4-913d-091b57f5bfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:36.853302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d7084-623d-42e4-913d-091b57f5bfeb None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:53:36.861660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dddf99-b0bb-46fb-8cc2-a799645ae125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:53:36.862801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dddf99-b0bb-46fb-8cc2-a799645ae125 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:36.863279 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dddf99-b0bb-46fb-8cc2-a799645ae125 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:36.863766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dddf99-b0bb-46fb-8cc2-a799645ae125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:36.880105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dddf99-b0bb-46fb-8cc2-a799645ae125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:53:36.882804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dddf99-b0bb-46fb-8cc2-a799645ae125 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:53:36.900545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 759/1513] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:53:37.564719 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107936f-f3d0-47ba-a8bf-6380586df5e2 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:37.565308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107936f-f3d0-47ba-a8bf-6380586df5e2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:37.565308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107936f-f3d0-47ba-a8bf-6380586df5e2 None None] PATH_INFO: `/v3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:37.566633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 755/1514] 10.222.0.87 () {54 vars in 925 bytes} [Fri May 13 13:53:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:53:37.574295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b30223-ca46-4060-b4bd-f3fae3a0441a None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:37.574845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b30223-ca46-4060-b4bd-f3fae3a0441a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:37.574845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b30223-ca46-4060-b4bd-f3fae3a0441a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:37.601481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-00b30223-ca46-4060-b4bd-f3fae3a0441a None None] MFA Rules not processed for user `4ae9a6f1ac304e21b8d3115f2a0baf88`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:53:37.601481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-00b30223-ca46-4060-b4bd-f3fae3a0441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:37.692519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 760/1515] 10.222.0.87 () {58 vars in 1015 bytes} [Fri May 13 13:53:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:53:37.936265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a084c-94d5-466e-9151-c307efb3fabe None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:37.936555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a084c-94d5-466e-9151-c307efb3fabe None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:37.937366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a084c-94d5-466e-9151-c307efb3fabe None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:37.939121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 756/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:53:37.944372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f79490-f962-43b5-849d-4cd9c2f4f034 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:37.944660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f79490-f962-43b5-849d-4cd9c2f4f034 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:37.944941 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f79490-f962-43b5-849d-4cd9c2f4f034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:37.964220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-73f79490-f962-43b5-849d-4cd9c2f4f034 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:53:37.965471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-73f79490-f962-43b5-849d-4cd9c2f4f034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:37.978746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 761/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:53:38.136037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a084c-94d5-466e-9151-c307efb3fabe None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:53:38.155585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a084c-94d5-466e-9151-c307efb3fabe None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:53:38.157499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fb485-4baf-4655-89c6-d5335437c9e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:53:38.158152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fb485-4baf-4655-89c6-d5335437c9e7 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:38.158431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fb485-4baf-4655-89c6-d5335437c9e7 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:38.158814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fb485-4baf-4655-89c6-d5335437c9e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:38.168495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fb485-4baf-4655-89c6-d5335437c9e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:53:38.170177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fb485-4baf-4655-89c6-d5335437c9e7 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:53:38.181039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 757/1518] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:53:47.676169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fcc8b-c163-4ad4-a922-fb15da6540de None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:47.676713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fcc8b-c163-4ad4-a922-fb15da6540de None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:47.676713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fcc8b-c163-4ad4-a922-fb15da6540de None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:47.678161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 762/1519] 10.222.0.79 () {54 vars in 960 bytes} [Fri May 13 13:53:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:53:47.685980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd21ca4-6b2c-42ed-bc07-5b980833ed86 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:47.686624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd21ca4-6b2c-42ed-bc07-5b980833ed86 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:47.686624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd21ca4-6b2c-42ed-bc07-5b980833ed86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:47.706040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1cd21ca4-6b2c-42ed-bc07-5b980833ed86 None None] MFA Rules not processed for user `4ae9a6f1ac304e21b8d3115f2a0baf88`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:53:47.707325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1cd21ca4-6b2c-42ed-bc07-5b980833ed86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:47.716265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 758/1520] 10.222.0.79 () {58 vars in 1057 bytes} [Fri May 13 13:53:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:53:47.724768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fcc8b-c163-4ad4-a922-fb15da6540de None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:53:47.743437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fcc8b-c163-4ad4-a922-fb15da6540de None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:53:47.749920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e745d36-4900-4f66-b562-071890396bba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ae9a6f1ac304e21b8d3115f2a0baf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:53:47.750580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e745d36-4900-4f66-b562-071890396bba service placement] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:47.750894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e745d36-4900-4f66-b562-071890396bba service placement] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:47.751168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e745d36-4900-4f66-b562-071890396bba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:47.760323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e745d36-4900-4f66-b562-071890396bba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:53:47.762004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e745d36-4900-4f66-b562-071890396bba service placement] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:53:47.773392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 763/1521] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:53:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:53:49.620172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e63bc-0799-48db-b92e-22883a838906 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:49.620806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e63bc-0799-48db-b92e-22883a838906 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:49.620806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e63bc-0799-48db-b92e-22883a838906 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:49.622124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:53:49.627991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0892e9-bebd-4b0d-ae90-96f508408a50 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:49.628526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0892e9-bebd-4b0d-ae90-96f508408a50 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:49.628680 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0892e9-bebd-4b0d-ae90-96f508408a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:49.648502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-5e0892e9-bebd-4b0d-ae90-96f508408a50 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:53:49.649785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5e0892e9-bebd-4b0d-ae90-96f508408a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:49.659588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:53:49.830550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e63bc-0799-48db-b92e-22883a838906 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:53:49.845746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e63bc-0799-48db-b92e-22883a838906 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:53:49.851546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6380a1-e367-45cb-ba36-a28940e62412 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:53:49.852216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6380a1-e367-45cb-ba36-a28940e62412 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:49.852553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6380a1-e367-45cb-ba36-a28940e62412 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:49.852826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6380a1-e367-45cb-ba36-a28940e62412 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:49.861811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6380a1-e367-45cb-ba36-a28940e62412 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:53:49.863502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6380a1-e367-45cb-ba36-a28940e62412 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:53:49.874426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 760/1524] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:53:50.855675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab581b-5fdc-4a0c-a0f1-3d532248e73a None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:50.856216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab581b-5fdc-4a0c-a0f1-3d532248e73a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:50.856216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab581b-5fdc-4a0c-a0f1-3d532248e73a None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:50.857569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:53:50.864128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b3a04-9828-451e-894f-b3e55fcda7cc None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:50.864705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b3a04-9828-451e-894f-b3e55fcda7cc None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:50.864705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b3a04-9828-451e-894f-b3e55fcda7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:50.887636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-2b6b3a04-9828-451e-894f-b3e55fcda7cc None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:53:50.889694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2b6b3a04-9828-451e-894f-b3e55fcda7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:50.901660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:53:51.062205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab581b-5fdc-4a0c-a0f1-3d532248e73a None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:53:51.083209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab581b-5fdc-4a0c-a0f1-3d532248e73a None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:53:51.089092 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11f620-e70f-41a5-aaa4-de1295fd2abb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:53:51.089809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11f620-e70f-41a5-aaa4-de1295fd2abb service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:51.090132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11f620-e70f-41a5-aaa4-de1295fd2abb service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:51.090449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11f620-e70f-41a5-aaa4-de1295fd2abb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:51.102522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11f620-e70f-41a5-aaa4-de1295fd2abb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:53:51.104154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11f620-e70f-41a5-aaa4-de1295fd2abb service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:53:51.114699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 766/1527] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:53:55.701181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa1bef-17f1-49f2-b8f3-5acfcc9df36b None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:55.702071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa1bef-17f1-49f2-b8f3-5acfcc9df36b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:55.702071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa1bef-17f1-49f2-b8f3-5acfcc9df36b None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:55.703661 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:53:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:53:55.709818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd1cff-08fd-446c-8321-ebd75312c8fc None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:55.710416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd1cff-08fd-446c-8321-ebd75312c8fc None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:55.710416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd1cff-08fd-446c-8321-ebd75312c8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:55.730365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-c2bd1cff-08fd-446c-8321-ebd75312c8fc None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:53:55.731646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c2bd1cff-08fd-446c-8321-ebd75312c8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:55.741045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:53:55.783757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa1bef-17f1-49f2-b8f3-5acfcc9df36b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:53:55.785547 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-73fa1bef-17f1-49f2-b8f3-5acfcc9df36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:55.849817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa1bef-17f1-49f2-b8f3-5acfcc9df36b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:53:55.853527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685904eb-7546-4c3f-b279-262ed1d503b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:53:55.854599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685904eb-7546-4c3f-b279-262ed1d503b4 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:55.854599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685904eb-7546-4c3f-b279-262ed1d503b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:55.858516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685904eb-7546-4c3f-b279-262ed1d503b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:55.866828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685904eb-7546-4c3f-b279-262ed1d503b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:53:55.868288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685904eb-7546-4c3f-b279-262ed1d503b4 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:53:55.878259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 763/1530] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:53:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:53:58.217103 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b70a2c-6ce7-404f-ae96-f1f3dc382037 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:58.217647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b70a2c-6ce7-404f-ae96-f1f3dc382037 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:58.217739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b70a2c-6ce7-404f-ae96-f1f3dc382037 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:58.219204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:53:58.224636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b6b9f-7a76-45aa-b498-a818067c6f44 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:58.225125 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b6b9f-7a76-45aa-b498-a818067c6f44 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:58.225222 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b6b9f-7a76-45aa-b498-a818067c6f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:58.243848 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-210b6b9f-7a76-45aa-b498-a818067c6f44 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:53:58.244979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-210b6b9f-7a76-45aa-b498-a818067c6f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:58.253994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:53:58.292264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b70a2c-6ce7-404f-ae96-f1f3dc382037 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:53:58.294630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d9b70a2c-6ce7-404f-ae96-f1f3dc382037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:53:58.309433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b70a2c-6ce7-404f-ae96-f1f3dc382037 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:53:58.314196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d94e6-1987-4a1a-930e-6c246e9c88b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:53:58.314968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d94e6-1987-4a1a-930e-6c246e9c88b4 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:53:58.315333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d94e6-1987-4a1a-930e-6c246e9c88b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:53:58.315617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d94e6-1987-4a1a-930e-6c246e9c88b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:53:58.325335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d94e6-1987-4a1a-930e-6c246e9c88b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:53:58.327511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d94e6-1987-4a1a-930e-6c246e9c88b4 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:53:58.337474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 769/1533] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:54:02.704363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15389f6-30dc-43d2-87fa-2fdb802b1482 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:54:02.704947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15389f6-30dc-43d2-87fa-2fdb802b1482 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:54:02.704947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15389f6-30dc-43d2-87fa-2fdb802b1482 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:54:02.706508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:54:02.712202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e090d-7a75-4716-81b8-0cad0a7e6356 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:54:02.712719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e090d-7a75-4716-81b8-0cad0a7e6356 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:54:02.712836 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e090d-7a75-4716-81b8-0cad0a7e6356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:54:02.731514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-023e090d-7a75-4716-81b8-0cad0a7e6356 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:54:02.732656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-023e090d-7a75-4716-81b8-0cad0a7e6356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:54:02.742757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:54:03.067620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15389f6-30dc-43d2-87fa-2fdb802b1482 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:54:03.069542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b15389f6-30dc-43d2-87fa-2fdb802b1482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:54:03.135020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15389f6-30dc-43d2-87fa-2fdb802b1482 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:54:03.139127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b45fc-f7fc-44df-be35-e6523a93c2d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:54:03.140024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b45fc-f7fc-44df-be35-e6523a93c2d2 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:54:03.140307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b45fc-f7fc-44df-be35-e6523a93c2d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:54:03.140583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b45fc-f7fc-44df-be35-e6523a93c2d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:54:03.151765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b45fc-f7fc-44df-be35-e6523a93c2d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:54:03.153225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b45fc-f7fc-44df-be35-e6523a93c2d2 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:54:03.164156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 766/1536] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:54:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:54:05.574466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0d556-6490-436f-85b2-cd921b6b39e1 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:54:05.575095 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0d556-6490-436f-85b2-cd921b6b39e1 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:54:05.575195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0d556-6490-436f-85b2-cd921b6b39e1 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:54:05.577637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:54:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:54:05.584285 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4eca7-6428-49e3-95ff-74c03f24cd59 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:54:05.584853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4eca7-6428-49e3-95ff-74c03f24cd59 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:54:05.584853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4eca7-6428-49e3-95ff-74c03f24cd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:54:05.603662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-06e4eca7-6428-49e3-95ff-74c03f24cd59 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:54:05.604924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-06e4eca7-6428-49e3-95ff-74c03f24cd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:54:05.616320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:54:05.898373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0d556-6490-436f-85b2-cd921b6b39e1 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:54:05.900693 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5be0d556-6490-436f-85b2-cd921b6b39e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:54:05.914207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0d556-6490-436f-85b2-cd921b6b39e1 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:54:05.917760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867445b-a65d-458e-b2f3-9d601746ca07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:54:05.918407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867445b-a65d-458e-b2f3-9d601746ca07 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:54:05.918687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867445b-a65d-458e-b2f3-9d601746ca07 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:54:05.918999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867445b-a65d-458e-b2f3-9d601746ca07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:54:05.929265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867445b-a65d-458e-b2f3-9d601746ca07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:54:05.930751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867445b-a65d-458e-b2f3-9d601746ca07 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:54:05.942240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 772/1539] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:40.166058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df7121-a276-46d8-90ae-22cb7b945e54 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.167730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df7121-a276-46d8-90ae-22cb7b945e54 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.168526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df7121-a276-46d8-90ae-22cb7b945e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.217042 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-16df7121-a276-46d8-90ae-22cb7b945e54 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:40.219918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-16df7121-a276-46d8-90ae-22cb7b945e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:40.253772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 768/1540] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:40.269496 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6066b-4536-4388-a1a8-c7d8cebe5574 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.270393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6066b-4536-4388-a1a8-c7d8cebe5574 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.271222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6066b-4536-4388-a1a8-c7d8cebe5574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.320100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-46e6066b-4536-4388-a1a8-c7d8cebe5574 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:40.322195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-46e6066b-4536-4388-a1a8-c7d8cebe5574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:40.344582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 773/1541] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:40.352946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df7121-a276-46d8-90ae-22cb7b945e54 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:40.376715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df7121-a276-46d8-90ae-22cb7b945e54 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:40.394311 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407c6a1-10e3-4f69-ab3b-41d3a9bc90df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:40.395524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407c6a1-10e3-4f69-ab3b-41d3a9bc90df admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.396019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407c6a1-10e3-4f69-ab3b-41d3a9bc90df admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.396480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407c6a1-10e3-4f69-ab3b-41d3a9bc90df admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.397855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407c6a1-10e3-4f69-ab3b-41d3a9bc90df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:40.398373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407c6a1-10e3-4f69-ab3b-41d3a9bc90df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:40.401494 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407c6a1-10e3-4f69-ab3b-41d3a9bc90df admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:40.426481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 769/1542] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:55:40.435963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6066b-4536-4388-a1a8-c7d8cebe5574 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:40.462881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6066b-4536-4388-a1a8-c7d8cebe5574 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:40.480654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777de88e-f236-478c-a3d7-f298b5ac9694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:40.482265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777de88e-f236-478c-a3d7-f298b5ac9694 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.482813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777de88e-f236-478c-a3d7-f298b5ac9694 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.483278 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777de88e-f236-478c-a3d7-f298b5ac9694 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.484533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777de88e-f236-478c-a3d7-f298b5ac9694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:40.488336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777de88e-f236-478c-a3d7-f298b5ac9694 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:40.559083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5428ee-30c7-4610-9273-34f1ed7d1fff None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.559808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5428ee-30c7-4610-9273-34f1ed7d1fff None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.560271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5428ee-30c7-4610-9273-34f1ed7d1fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.577682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 774/1543] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:40 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:40.583578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777de88e-f236-478c-a3d7-f298b5ac9694 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:40.631420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-0e5428ee-30c7-4610-9273-34f1ed7d1fff None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:40.634019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0e5428ee-30c7-4610-9273-34f1ed7d1fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:40.794100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777de88e-f236-478c-a3d7-f298b5ac9694 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:40.796956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 770/1544] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:40.798600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a51d2-4f79-4490-b8ad-05d93e500425 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.799676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a51d2-4f79-4490-b8ad-05d93e500425 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.800187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a51d2-4f79-4490-b8ad-05d93e500425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.813540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca54b1-c0a7-4c7d-bfa7-0d501997e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:40.815530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca54b1-c0a7-4c7d-bfa7-0d501997e411 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.816021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca54b1-c0a7-4c7d-bfa7-0d501997e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.816503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca54b1-c0a7-4c7d-bfa7-0d501997e411 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.817818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca54b1-c0a7-4c7d-bfa7-0d501997e411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:40.819688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: WARNING py.warnings [None req-c2ca54b1-c0a7-4c7d-bfa7-0d501997e411 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:55:40.819688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: warnings.warn(msg) May 13 13:55:40.819688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: May 13 13:55:40.821967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca54b1-c0a7-4c7d-bfa7-0d501997e411 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:40.837404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-977a51d2-4f79-4490-b8ad-05d93e500425 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:40.839798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-977a51d2-4f79-4490-b8ad-05d93e500425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:40.851657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 771/1545] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:40.853637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d851085-0ead-441b-8e0f-b3d1f44cb4cf None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.854201 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d851085-0ead-441b-8e0f-b3d1f44cb4cf None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.854942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d851085-0ead-441b-8e0f-b3d1f44cb4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.869408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 775/1546] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:40 2022] POST /identity/v3/users => generated 287 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:55:40.871305 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d282b-3226-4209-878a-8d8542025973 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.872565 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d282b-3226-4209-878a-8d8542025973 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.875311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d282b-3226-4209-878a-8d8542025973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.902388 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-2d851085-0ead-441b-8e0f-b3d1f44cb4cf None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:40.905914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2d851085-0ead-441b-8e0f-b3d1f44cb4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:40.913072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-927d282b-3226-4209-878a-8d8542025973 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:40.915869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-927d282b-3226-4209-878a-8d8542025973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:40.927262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 772/1547] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:40.931337 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121bb06-fe79-43e6-82a2-8ef4f9aaff89 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.932403 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121bb06-fe79-43e6-82a2-8ef4f9aaff89 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.933816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121bb06-fe79-43e6-82a2-8ef4f9aaff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.937934 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 776/1548] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:40.943341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9f8cf-56ff-41e6-a7ba-e1c8bec7e70e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:40.944740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9f8cf-56ff-41e6-a7ba-e1c8bec7e70e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:40.945262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9f8cf-56ff-41e6-a7ba-e1c8bec7e70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:40.976022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b121bb06-fe79-43e6-82a2-8ef4f9aaff89 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:40.977907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b121bb06-fe79-43e6-82a2-8ef4f9aaff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:40.984411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-eac9f8cf-56ff-41e6-a7ba-e1c8bec7e70e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:40.987771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-eac9f8cf-56ff-41e6-a7ba-e1c8bec7e70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:41.005256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 773/1549] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.006609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121bb06-fe79-43e6-82a2-8ef4f9aaff89 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.016356 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 777/1550] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.018049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3179f9b-0e34-484b-a3cf-262f4825890e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.018569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3179f9b-0e34-484b-a3cf-262f4825890e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.019106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3179f9b-0e34-484b-a3cf-262f4825890e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.025805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121bb06-fe79-43e6-82a2-8ef4f9aaff89 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.040413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed1db0-1e9d-4b80-ba84-0bf8f9d14d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.041491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed1db0-1e9d-4b80-ba84-0bf8f9d14d2a admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.041997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed1db0-1e9d-4b80-ba84-0bf8f9d14d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.042517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed1db0-1e9d-4b80-ba84-0bf8f9d14d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.043489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed1db0-1e9d-4b80-ba84-0bf8f9d14d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.045430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed1db0-1e9d-4b80-ba84-0bf8f9d14d2a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.056923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a3179f9b-0e34-484b-a3cf-262f4825890e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:41.058241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a3179f9b-0e34-484b-a3cf-262f4825890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:41.063093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 774/1551] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:41.063093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fe0fe-9815-4884-8b84-845ef51b6371 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.063552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fe0fe-9815-4884-8b84-845ef51b6371 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.063552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fe0fe-9815-4884-8b84-845ef51b6371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.074879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 778/1552] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.076716 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3179f9b-0e34-484b-a3cf-262f4825890e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.086270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-316fe0fe-9815-4884-8b84-845ef51b6371 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:41.087537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-316fe0fe-9815-4884-8b84-845ef51b6371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:41.093353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3179f9b-0e34-484b-a3cf-262f4825890e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.097468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 775/1553] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.098315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fe0fe-9815-4884-8b84-845ef51b6371 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.101882 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff3910-6a2e-4fbd-bb95-a6e3a8318632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.103055 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3910-6a2e-4fbd-bb95-a6e3a8318632 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.103654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3910-6a2e-4fbd-bb95-a6e3a8318632 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.104042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff3910-6a2e-4fbd-bb95-a6e3a8318632 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.104967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff3910-6a2e-4fbd-bb95-a6e3a8318632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:41.105472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff3910-6a2e-4fbd-bb95-a6e3a8318632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.106952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: WARNING py.warnings [None req-bcff3910-6a2e-4fbd-bb95-a6e3a8318632 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 13:55:41.106952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: warnings.warn(msg) May 13 13:55:41.106952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: May 13 13:55:41.107717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff3910-6a2e-4fbd-bb95-a6e3a8318632 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.110695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fe0fe-9815-4884-8b84-845ef51b6371 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.116794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a61c72-e8a4-4700-b5bb-85e74792cf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.117459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a61c72-e8a4-4700-b5bb-85e74792cf54 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.117735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a61c72-e8a4-4700-b5bb-85e74792cf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.118102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a61c72-e8a4-4700-b5bb-85e74792cf54 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.118866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a61c72-e8a4-4700-b5bb-85e74792cf54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:41.119313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a61c72-e8a4-4700-b5bb-85e74792cf54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.121337 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a61c72-e8a4-4700-b5bb-85e74792cf54 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.127889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 779/1554] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:55:41.130929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10251f3a-4609-449c-a363-e7c9eccdf38b None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.131441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10251f3a-4609-449c-a363-e7c9eccdf38b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.131961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10251f3a-4609-449c-a363-e7c9eccdf38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.149912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 776/1555] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:55:41.149912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67073c4-6743-4da4-b71a-ed6429eb26b5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.149912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67073c4-6743-4da4-b71a-ed6429eb26b5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.149912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67073c4-6743-4da4-b71a-ed6429eb26b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.166903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-10251f3a-4609-449c-a363-e7c9eccdf38b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:41.169097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-10251f3a-4609-449c-a363-e7c9eccdf38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:41.178192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f67073c4-6743-4da4-b71a-ed6429eb26b5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:41.179456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f67073c4-6743-4da4-b71a-ed6429eb26b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:41.183984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 780/1556] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.185944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10251f3a-4609-449c-a363-e7c9eccdf38b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.193601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 777/1557] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.200156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67073c4-6743-4da4-b71a-ed6429eb26b5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.204319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10251f3a-4609-449c-a363-e7c9eccdf38b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.210184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa9a18-ba91-456e-acbe-ff6f15b8e1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.210981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa9a18-ba91-456e-acbe-ff6f15b8e1ca admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.211273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa9a18-ba91-456e-acbe-ff6f15b8e1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.211544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa9a18-ba91-456e-acbe-ff6f15b8e1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.212275 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa9a18-ba91-456e-acbe-ff6f15b8e1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.213548 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67073c4-6743-4da4-b71a-ed6429eb26b5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.213885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa9a18-ba91-456e-acbe-ff6f15b8e1ca admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.218657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07bccd-fc37-4e09-b963-a8f65e4c858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.219434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07bccd-fc37-4e09-b963-a8f65e4c858e admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.219781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07bccd-fc37-4e09-b963-a8f65e4c858e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.220216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07bccd-fc37-4e09-b963-a8f65e4c858e admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.221649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07bccd-fc37-4e09-b963-a8f65e4c858e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:41.221949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07bccd-fc37-4e09-b963-a8f65e4c858e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.223590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07bccd-fc37-4e09-b963-a8f65e4c858e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.226223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 781/1558] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:41.227187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa9a18-ba91-456e-acbe-ff6f15b8e1ca admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.238196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa9a18-ba91-456e-acbe-ff6f15b8e1ca admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.240153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 778/1559] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:55:41.241160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07bccd-fc37-4e09-b963-a8f65e4c858e admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.245692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780f8b6-467f-4949-9e0b-d7dc66004938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.246620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780f8b6-467f-4949-9e0b-d7dc66004938 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.247111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780f8b6-467f-4949-9e0b-d7dc66004938 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.247569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780f8b6-467f-4949-9e0b-d7dc66004938 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.248778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780f8b6-467f-4949-9e0b-d7dc66004938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:41.249195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780f8b6-467f-4949-9e0b-d7dc66004938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.251807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780f8b6-467f-4949-9e0b-d7dc66004938 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.256316 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07bccd-fc37-4e09-b963-a8f65e4c858e admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.263150 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f23260-33a4-407d-9de4-2ebbc849e6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.264152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f23260-33a4-407d-9de4-2ebbc849e6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.264615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f23260-33a4-407d-9de4-2ebbc849e6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.265069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f23260-33a4-407d-9de4-2ebbc849e6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.266219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f23260-33a4-407d-9de4-2ebbc849e6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.267767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WARNING py.warnings [None req-11f23260-33a4-407d-9de4-2ebbc849e6b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:55:41.267767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: warnings.warn(msg) May 13 13:55:41.267767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: May 13 13:55:41.269771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f23260-33a4-407d-9de4-2ebbc849e6b7 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.278959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 782/1560] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:55:41.278959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780f8b6-467f-4949-9e0b-d7dc66004938 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.284565 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780f8b6-467f-4949-9e0b-d7dc66004938 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.289652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0c10a-2603-4713-8c07-1cf4f675ac8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.290311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0c10a-2603-4713-8c07-1cf4f675ac8b admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.290623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0c10a-2603-4713-8c07-1cf4f675ac8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.290937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0c10a-2603-4713-8c07-1cf4f675ac8b admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.291692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0c10a-2603-4713-8c07-1cf4f675ac8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.293614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0c10a-2603-4713-8c07-1cf4f675ac8b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.307444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 779/1561] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:41.313032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83431a00-01d4-4609-981b-f569b4975f97 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.313407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83431a00-01d4-4609-981b-f569b4975f97 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.313778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83431a00-01d4-4609-981b-f569b4975f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.330345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 783/1562] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:41.332575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67a2e8-f7ee-4b42-8950-f299eb6458a7 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.333061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67a2e8-f7ee-4b42-8950-f299eb6458a7 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.333532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67a2e8-f7ee-4b42-8950-f299eb6458a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.344411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-83431a00-01d4-4609-981b-f569b4975f97 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:41.345628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-83431a00-01d4-4609-981b-f569b4975f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:41.355394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-cd67a2e8-f7ee-4b42-8950-f299eb6458a7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:41.356589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-cd67a2e8-f7ee-4b42-8950-f299eb6458a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:41.409701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 780/1563] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.411285 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a411013-43f1-433a-be8b-6fe291e94cc2 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.411574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a411013-43f1-433a-be8b-6fe291e94cc2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.411856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a411013-43f1-433a-be8b-6fe291e94cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.424374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 784/1564] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.425546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67a2e8-f7ee-4b42-8950-f299eb6458a7 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.435050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-2a411013-43f1-433a-be8b-6fe291e94cc2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:41.436913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2a411013-43f1-433a-be8b-6fe291e94cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:41.440168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67a2e8-f7ee-4b42-8950-f299eb6458a7 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.447238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10269828-01b6-4a08-86ec-21fa38a93534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.448650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10269828-01b6-4a08-86ec-21fa38a93534 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.453040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10269828-01b6-4a08-86ec-21fa38a93534 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.453190 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 781/1565] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.453321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10269828-01b6-4a08-86ec-21fa38a93534 admin admin] PATH_INFO: `/v3/projects/0887bb8970724681908617358b2e9831/users/da08dcccc94842ba9911450666c4fa72/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.453427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a411013-43f1-433a-be8b-6fe291e94cc2 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.465420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a411013-43f1-433a-be8b-6fe291e94cc2 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.470677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bd375-e301-4db3-92dc-eb9fc44fee88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.471727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10269828-01b6-4a08-86ec-21fa38a93534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0887bb8970724681908617358b2e9831, user_id=da08dcccc94842ba9911450666c4fa72, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.472099 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bd375-e301-4db3-92dc-eb9fc44fee88 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.472378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bd375-e301-4db3-92dc-eb9fc44fee88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.472651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bd375-e301-4db3-92dc-eb9fc44fee88 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.473345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bd375-e301-4db3-92dc-eb9fc44fee88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.475122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10269828-01b6-4a08-86ec-21fa38a93534 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.475356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bd375-e301-4db3-92dc-eb9fc44fee88 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.492521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 785/1566] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:41 2022] PUT /identity/v3/projects/0887bb8970724681908617358b2e9831/users/da08dcccc94842ba9911450666c4fa72/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:41.493657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10269828-01b6-4a08-86ec-21fa38a93534 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.516083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 782/1567] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:41.517147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bd375-e301-4db3-92dc-eb9fc44fee88 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.574441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10269828-01b6-4a08-86ec-21fa38a93534 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.600212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bd375-e301-4db3-92dc-eb9fc44fee88 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.606076 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1983f9d-afef-46ad-b9fd-ce77f1d76c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.606942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1983f9d-afef-46ad-b9fd-ce77f1d76c8f admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.607382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1983f9d-afef-46ad-b9fd-ce77f1d76c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.607766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1983f9d-afef-46ad-b9fd-ce77f1d76c8f admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.608610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1983f9d-afef-46ad-b9fd-ce77f1d76c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.610905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1983f9d-afef-46ad-b9fd-ce77f1d76c8f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.637032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fd720-053f-465f-999a-3b89bc095e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.637913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fd720-053f-465f-999a-3b89bc095e85 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.638845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fd720-053f-465f-999a-3b89bc095e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.638845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fd720-053f-465f-999a-3b89bc095e85 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.639072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 783/1568] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:41.639442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fd720-053f-465f-999a-3b89bc095e85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.639948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1983f9d-afef-46ad-b9fd-ce77f1d76c8f admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.641266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fd720-053f-465f-999a-3b89bc095e85 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.651738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1983f9d-afef-46ad-b9fd-ce77f1d76c8f admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.658019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82933e5d-bade-4356-af3c-b5c9d00617c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.659483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82933e5d-bade-4356-af3c-b5c9d00617c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.659897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82933e5d-bade-4356-af3c-b5c9d00617c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.660370 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82933e5d-bade-4356-af3c-b5c9d00617c9 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.661173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82933e5d-bade-4356-af3c-b5c9d00617c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.663175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82933e5d-bade-4356-af3c-b5c9d00617c9 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.671625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 786/1569] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/projects => generated 393 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:41.674171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3417a-41f1-4652-ae7e-659c9e7557e3 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.674755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3417a-41f1-4652-ae7e-659c9e7557e3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.675922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3417a-41f1-4652-ae7e-659c9e7557e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.690684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 784/1570] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/users => generated 303 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:41.692376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82933e5d-bade-4356-af3c-b5c9d00617c9 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.702633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-97b3417a-41f1-4652-ae7e-659c9e7557e3 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:41.704318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-97b3417a-41f1-4652-ae7e-659c9e7557e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:41.705694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82933e5d-bade-4356-af3c-b5c9d00617c9 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.773495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8910f5-cf88-4ca7-96a3-92dde1cb80f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.774309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8910f5-cf88-4ca7-96a3-92dde1cb80f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.774725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8910f5-cf88-4ca7-96a3-92dde1cb80f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.775137 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8910f5-cf88-4ca7-96a3-92dde1cb80f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.775504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 787/1571] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.776252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8910f5-cf88-4ca7-96a3-92dde1cb80f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:41.776504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3417a-41f1-4652-ae7e-659c9e7557e3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.776866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8910f5-cf88-4ca7-96a3-92dde1cb80f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.779898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8910f5-cf88-4ca7-96a3-92dde1cb80f3 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.791536 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3417a-41f1-4652-ae7e-659c9e7557e3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.796250 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 785/1572] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:55:41.796584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f645455-9bf6-427d-966f-be26e301e516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.797475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f645455-9bf6-427d-966f-be26e301e516 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.797890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f645455-9bf6-427d-966f-be26e301e516 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.798176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f645455-9bf6-427d-966f-be26e301e516 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.798568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c83b4d-20bf-42bd-807f-c0d50a244b4d None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.799519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f645455-9bf6-427d-966f-be26e301e516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:41.799699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c83b4d-20bf-42bd-807f-c0d50a244b4d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.799939 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f645455-9bf6-427d-966f-be26e301e516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.800092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c83b4d-20bf-42bd-807f-c0d50a244b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.801645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f645455-9bf6-427d-966f-be26e301e516 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.815687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 788/1573] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:55:41.816609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f645455-9bf6-427d-966f-be26e301e516 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.828283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f645455-9bf6-427d-966f-be26e301e516 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.833984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c3735-ac45-4019-845b-ca267daa252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.834909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c3735-ac45-4019-845b-ca267daa252e admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.835204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c3735-ac45-4019-845b-ca267daa252e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.835510 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c3735-ac45-4019-845b-ca267daa252e admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.836198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c3735-ac45-4019-845b-ca267daa252e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.838303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e4c83b4d-20bf-42bd-807f-c0d50a244b4d None None] MFA Rules not processed for user `da08dcccc94842ba9911450666c4fa72`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:41.838762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c3735-ac45-4019-845b-ca267daa252e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.839756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e4c83b4d-20bf-42bd-807f-c0d50a244b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:41.870648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 789/1574] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/users => generated 289 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:41.872295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c3735-ac45-4019-845b-ca267daa252e admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.891930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c3735-ac45-4019-845b-ca267daa252e admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.900883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adecb81-cdba-4691-9097-e70663fba4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.902005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adecb81-cdba-4691-9097-e70663fba4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.902930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adecb81-cdba-4691-9097-e70663fba4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.903465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adecb81-cdba-4691-9097-e70663fba4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.904311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adecb81-cdba-4691-9097-e70663fba4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.906969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adecb81-cdba-4691-9097-e70663fba4a4 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.922750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 790/1575] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:41.923682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adecb81-cdba-4691-9097-e70663fba4a4 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.938096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 786/1576] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:41.942746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adecb81-cdba-4691-9097-e70663fba4a4 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.942870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c83b4d-20bf-42bd-807f-c0d50a244b4d None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.947110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8d8eb-ce48-4611-b6a2-1d85f9c55905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.947796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8d8eb-ce48-4611-b6a2-1d85f9c55905 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.948087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8d8eb-ce48-4611-b6a2-1d85f9c55905 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.948487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8d8eb-ce48-4611-b6a2-1d85f9c55905 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.949255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8d8eb-ce48-4611-b6a2-1d85f9c55905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.951551 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8d8eb-ce48-4611-b6a2-1d85f9c55905 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.954827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c83b4d-20bf-42bd-807f-c0d50a244b4d None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.960492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46837510-f0c2-477e-82c5-8756a86a4d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.961336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46837510-f0c2-477e-82c5-8756a86a4d91 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.961734 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46837510-f0c2-477e-82c5-8756a86a4d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.962192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46837510-f0c2-477e-82c5-8756a86a4d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.963370 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46837510-f0c2-477e-82c5-8756a86a4d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.965592 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46837510-f0c2-477e-82c5-8756a86a4d91 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:41.976544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 791/1577] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/users => generated 297 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:41.976544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8d8eb-ce48-4611-b6a2-1d85f9c55905 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.980646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 787/1578] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:41.981937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46837510-f0c2-477e-82c5-8756a86a4d91 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:41.989742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8d8eb-ce48-4611-b6a2-1d85f9c55905 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.992261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46837510-f0c2-477e-82c5-8756a86a4d91 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:41.996279 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91fe4f-4f91-4c02-b9c2-9c705d0a8581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.997063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed3814-4c7a-4389-b224-f8c9848c11a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:41.997201 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91fe4f-4f91-4c02-b9c2-9c705d0a8581 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.997706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed3814-4c7a-4389-b224-f8c9848c11a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:41.997866 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91fe4f-4f91-4c02-b9c2-9c705d0a8581 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.998209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed3814-4c7a-4389-b224-f8c9848c11a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:41.998383 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91fe4f-4f91-4c02-b9c2-9c705d0a8581 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.998544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed3814-4c7a-4389-b224-f8c9848c11a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:41.999013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed3814-4c7a-4389-b224-f8c9848c11a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:41.999154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91fe4f-4f91-4c02-b9c2-9c705d0a8581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:41.999521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91fe4f-4f91-4c02-b9c2-9c705d0a8581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.000851 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed3814-4c7a-4389-b224-f8c9848c11a5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.001216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91fe4f-4f91-4c02-b9c2-9c705d0a8581 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.016461 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 792/1579] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:55:42.018332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91fe4f-4f91-4c02-b9c2-9c705d0a8581 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.034467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 788/1580] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:41 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:42.035497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed3814-4c7a-4389-b224-f8c9848c11a5 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.037527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91fe4f-4f91-4c02-b9c2-9c705d0a8581 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.098370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a58652-8faa-40c4-856b-7c5d58ef8561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.099021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a58652-8faa-40c4-856b-7c5d58ef8561 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.099435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a58652-8faa-40c4-856b-7c5d58ef8561 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.099718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a58652-8faa-40c4-856b-7c5d58ef8561 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.100443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a58652-8faa-40c4-856b-7c5d58ef8561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.102415 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a58652-8faa-40c4-856b-7c5d58ef8561 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.103249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed3814-4c7a-4389-b224-f8c9848c11a5 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.109083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac55110-526f-4607-a12d-b8cfbb4e9d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.109976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac55110-526f-4607-a12d-b8cfbb4e9d97 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.110265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac55110-526f-4607-a12d-b8cfbb4e9d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.110541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac55110-526f-4607-a12d-b8cfbb4e9d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.111939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac55110-526f-4607-a12d-b8cfbb4e9d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.113580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac55110-526f-4607-a12d-b8cfbb4e9d97 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.124975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 789/1581] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:42.126081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac55110-526f-4607-a12d-b8cfbb4e9d97 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.130808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 793/1582] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:42 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:42.131729 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a58652-8faa-40c4-856b-7c5d58ef8561 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.141547 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac55110-526f-4607-a12d-b8cfbb4e9d97 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.212002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a58652-8faa-40c4-856b-7c5d58ef8561 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.213390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b265f-a020-46ba-9687-82b720ac6dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.214274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b265f-a020-46ba-9687-82b720ac6dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.214514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b265f-a020-46ba-9687-82b720ac6dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.214899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b265f-a020-46ba-9687-82b720ac6dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.216064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b265f-a020-46ba-9687-82b720ac6dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.217748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b265f-a020-46ba-9687-82b720ac6dc6 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.224805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f393-f76b-45a5-ac6e-4c497b03471d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.225985 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f393-f76b-45a5-ac6e-4c497b03471d admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.226538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f393-f76b-45a5-ac6e-4c497b03471d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.227085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f393-f76b-45a5-ac6e-4c497b03471d admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.228596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f393-f76b-45a5-ac6e-4c497b03471d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.229494 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 790/1583] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:42.230553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b265f-a020-46ba-9687-82b720ac6dc6 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.232310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f393-f76b-45a5-ac6e-4c497b03471d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.241789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b265f-a020-46ba-9687-82b720ac6dc6 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.247059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d19f-af54-4d3e-9f07-de069728e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.248198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d19f-af54-4d3e-9f07-de069728e286 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.248735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d19f-af54-4d3e-9f07-de069728e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.249201 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d19f-af54-4d3e-9f07-de069728e286 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.250533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219d19f-af54-4d3e-9f07-de069728e286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.254235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219d19f-af54-4d3e-9f07-de069728e286 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.267125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 794/1584] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:42 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:42.268360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f393-f76b-45a5-ac6e-4c497b03471d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.276826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 791/1585] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:42.277815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d19f-af54-4d3e-9f07-de069728e286 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.288992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d19f-af54-4d3e-9f07-de069728e286 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.350330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f393-f76b-45a5-ac6e-4c497b03471d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.354512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71325568-f62f-48a8-8155-2379b4964e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.355264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71325568-f62f-48a8-8155-2379b4964e2c admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.355547 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71325568-f62f-48a8-8155-2379b4964e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.355871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71325568-f62f-48a8-8155-2379b4964e2c admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.356586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71325568-f62f-48a8-8155-2379b4964e2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.357204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61e6b6-a925-4c41-a153-201ce3abe682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.358206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61e6b6-a925-4c41-a153-201ce3abe682 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.358510 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71325568-f62f-48a8-8155-2379b4964e2c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.358659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61e6b6-a925-4c41-a153-201ce3abe682 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.359084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61e6b6-a925-4c41-a153-201ce3abe682 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.360226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da61e6b6-a925-4c41-a153-201ce3abe682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.363382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da61e6b6-a925-4c41-a153-201ce3abe682 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.380777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 795/1586] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:42.380777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61e6b6-a925-4c41-a153-201ce3abe682 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.394006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 792/1587] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:42 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:42.394006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71325568-f62f-48a8-8155-2379b4964e2c admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.398972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61e6b6-a925-4c41-a153-201ce3abe682 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.471640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71325568-f62f-48a8-8155-2379b4964e2c admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.477723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8fc92-d60e-4c69-8f49-b5e7554ecea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.478563 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8fc92-d60e-4c69-8f49-b5e7554ecea5 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.479002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8fc92-d60e-4c69-8f49-b5e7554ecea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.479380 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8fc92-d60e-4c69-8f49-b5e7554ecea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.480260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8fc92-d60e-4c69-8f49-b5e7554ecea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.482073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea3452-5719-496b-9493-1b1a55948331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.482299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8fc92-d60e-4c69-8f49-b5e7554ecea5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.483063 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea3452-5719-496b-9493-1b1a55948331 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.483441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea3452-5719-496b-9493-1b1a55948331 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.483839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea3452-5719-496b-9493-1b1a55948331 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.484660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea3452-5719-496b-9493-1b1a55948331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.487726 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea3452-5719-496b-9493-1b1a55948331 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.495116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 793/1588] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:42.496183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8fc92-d60e-4c69-8f49-b5e7554ecea5 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.510009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8fc92-d60e-4c69-8f49-b5e7554ecea5 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.516078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb822502-28c8-4229-b01d-57fb6e5cd50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.516865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822502-28c8-4229-b01d-57fb6e5cd50f admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.517085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 796/1589] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:42 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:42.517541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822502-28c8-4229-b01d-57fb6e5cd50f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.518032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822502-28c8-4229-b01d-57fb6e5cd50f admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.518300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea3452-5719-496b-9493-1b1a55948331 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.520224 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb822502-28c8-4229-b01d-57fb6e5cd50f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.523141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb822502-28c8-4229-b01d-57fb6e5cd50f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.535118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea3452-5719-496b-9493-1b1a55948331 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.543042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ead5a-aef4-413a-a120-87c64fd5d19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.544569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ead5a-aef4-413a-a120-87c64fd5d19a admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.545070 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ead5a-aef4-413a-a120-87c64fd5d19a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.545532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ead5a-aef4-413a-a120-87c64fd5d19a admin admin] PATH_INFO: `/v3/projects/ac9b3816e0cb4400919797c107065ad4/users/648ae92126944b50a22ad780e92df914/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.553609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 794/1590] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:42 2022] POST /identity/v3/users => generated 290 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:42.555056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb822502-28c8-4229-b01d-57fb6e5cd50f admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.569590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb822502-28c8-4229-b01d-57fb6e5cd50f admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.570497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ead5a-aef4-413a-a120-87c64fd5d19a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9b3816e0cb4400919797c107065ad4, user_id=648ae92126944b50a22ad780e92df914, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.574136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ead5a-aef4-413a-a120-87c64fd5d19a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.575052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b301-bb70-4d53-8c3d-a5ccfec0c5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.575784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b301-bb70-4d53-8c3d-a5ccfec0c5ee admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.576149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b301-bb70-4d53-8c3d-a5ccfec0c5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.576483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b301-bb70-4d53-8c3d-a5ccfec0c5ee admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.577451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7b301-bb70-4d53-8c3d-a5ccfec0c5ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.580017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7b301-bb70-4d53-8c3d-a5ccfec0c5ee admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.600017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 797/1591] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:42 2022] PUT /identity/v3/projects/ac9b3816e0cb4400919797c107065ad4/users/648ae92126944b50a22ad780e92df914/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:42.601119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ead5a-aef4-413a-a120-87c64fd5d19a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.614533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 795/1592] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:42 2022] POST /identity/v3/users => generated 293 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:42.614533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b301-bb70-4d53-8c3d-a5ccfec0c5ee admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.635263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b301-bb70-4d53-8c3d-a5ccfec0c5ee admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.690477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ead5a-aef4-413a-a120-87c64fd5d19a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.697303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f70e7e-cbaf-4343-8558-d8e7c9aa2a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.698348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f70e7e-cbaf-4343-8558-d8e7c9aa2a23 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.698837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f70e7e-cbaf-4343-8558-d8e7c9aa2a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.699280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f70e7e-cbaf-4343-8558-d8e7c9aa2a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.700507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f70e7e-cbaf-4343-8558-d8e7c9aa2a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.703333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f70e7e-cbaf-4343-8558-d8e7c9aa2a23 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.724312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 798/1593] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:42.724312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f70e7e-cbaf-4343-8558-d8e7c9aa2a23 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.739277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68bb71-5d08-428b-bfce-2ce1eb89ad15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.740726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68bb71-5d08-428b-bfce-2ce1eb89ad15 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.741173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68bb71-5d08-428b-bfce-2ce1eb89ad15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.741344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f70e7e-cbaf-4343-8558-d8e7c9aa2a23 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.741624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68bb71-5d08-428b-bfce-2ce1eb89ad15 admin admin] PATH_INFO: `/v3/projects/b95ab7620c68480195a6e9505fdacfec/users/c672faa2a3d849adb0898cb11b5e5038/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.748047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa66209-f177-46d3-9a6e-2bde5398c542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.748822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa66209-f177-46d3-9a6e-2bde5398c542 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.749163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa66209-f177-46d3-9a6e-2bde5398c542 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.749503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa66209-f177-46d3-9a6e-2bde5398c542 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.750344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa66209-f177-46d3-9a6e-2bde5398c542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.752562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa66209-f177-46d3-9a6e-2bde5398c542 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.761273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68bb71-5d08-428b-bfce-2ce1eb89ad15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95ab7620c68480195a6e9505fdacfec, user_id=c672faa2a3d849adb0898cb11b5e5038, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.762593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WARNING py.warnings [None req-2c68bb71-5d08-428b-bfce-2ce1eb89ad15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:55:42.762593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: warnings.warn(msg) May 13 13:55:42.762593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: May 13 13:55:42.767330 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68bb71-5d08-428b-bfce-2ce1eb89ad15 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.782357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 799/1594] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:42 2022] POST /identity/v3/users => generated 287 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:42.784568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa66209-f177-46d3-9a6e-2bde5398c542 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.791044 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 796/1595] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:42 2022] PUT /identity/v3/projects/b95ab7620c68480195a6e9505fdacfec/users/c672faa2a3d849adb0898cb11b5e5038/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:42.792318 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68bb71-5d08-428b-bfce-2ce1eb89ad15 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.800658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa66209-f177-46d3-9a6e-2bde5398c542 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.870795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68bb71-5d08-428b-bfce-2ce1eb89ad15 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.871064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7f514-0056-4221-9f90-dee5de66658f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.872338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7f514-0056-4221-9f90-dee5de66658f admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.872797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7f514-0056-4221-9f90-dee5de66658f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.873258 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7f514-0056-4221-9f90-dee5de66658f admin admin] PATH_INFO: `/v3/projects/ef87180052d643af940a8e2b4f21e46f/users/a37f5d5ddd9845fbaf7d5645c31b2d40/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.878299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581c016-83c5-4835-a992-b6c9678c86f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.879257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c016-83c5-4835-a992-b6c9678c86f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.879712 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c016-83c5-4835-a992-b6c9678c86f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.880154 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c016-83c5-4835-a992-b6c9678c86f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.881367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581c016-83c5-4835-a992-b6c9678c86f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.884178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581c016-83c5-4835-a992-b6c9678c86f4 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.896615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 797/1596] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:42.897477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581c016-83c5-4835-a992-b6c9678c86f4 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.901568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7f514-0056-4221-9f90-dee5de66658f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef87180052d643af940a8e2b4f21e46f, user_id=a37f5d5ddd9845fbaf7d5645c31b2d40, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.906837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7f514-0056-4221-9f90-dee5de66658f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.912259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581c016-83c5-4835-a992-b6c9678c86f4 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.921745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736dfe3-7a51-48d4-92dd-dc0686d405ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:42.923038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736dfe3-7a51-48d4-92dd-dc0686d405ad admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.924271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736dfe3-7a51-48d4-92dd-dc0686d405ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.924550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736dfe3-7a51-48d4-92dd-dc0686d405ad admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.925422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c736dfe3-7a51-48d4-92dd-dc0686d405ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:42.927637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c736dfe3-7a51-48d4-92dd-dc0686d405ad admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:42.932065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 800/1597] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:42 2022] PUT /identity/v3/projects/ef87180052d643af940a8e2b4f21e46f/users/a37f5d5ddd9845fbaf7d5645c31b2d40/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:42.934689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ef82f-25da-440f-a413-47f21bd387fc None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:42.935221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ef82f-25da-440f-a413-47f21bd387fc None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:42.935735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ef82f-25da-440f-a413-47f21bd387fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:42.945537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 798/1598] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:42.945537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736dfe3-7a51-48d4-92dd-dc0686d405ad admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:42.956871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736dfe3-7a51-48d4-92dd-dc0686d405ad admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:42.964307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-d03ef82f-25da-440f-a413-47f21bd387fc None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:42.966366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d03ef82f-25da-440f-a413-47f21bd387fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:43.025142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cd873-b784-4ee5-9d84-bbdc93be69a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.025945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cd873-b784-4ee5-9d84-bbdc93be69a7 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.026367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cd873-b784-4ee5-9d84-bbdc93be69a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.026794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cd873-b784-4ee5-9d84-bbdc93be69a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.027749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34cd873-b784-4ee5-9d84-bbdc93be69a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.029701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34cd873-b784-4ee5-9d84-bbdc93be69a7 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.041520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 799/1599] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:43.042354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cd873-b784-4ee5-9d84-bbdc93be69a7 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.053887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cd873-b784-4ee5-9d84-bbdc93be69a7 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.059120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6eedfb-2876-4d92-9db8-9ce9fec992b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.060170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 801/1600] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:43.060692 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6eedfb-2876-4d92-9db8-9ce9fec992b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.061064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6eedfb-2876-4d92-9db8-9ce9fec992b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.061427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6eedfb-2876-4d92-9db8-9ce9fec992b3 admin admin] PATH_INFO: `/v3/projects/ef89791b177f4bf0ba916f133c79f33a/users/20fd230cc30f4980bedebc75c620d707/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.061604 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ef82f-25da-440f-a413-47f21bd387fc None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.077231 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ef82f-25da-440f-a413-47f21bd387fc None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.084485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b1e08-c248-49fd-ac99-e32533afa81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.085816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1e08-c248-49fd-ac99-e32533afa81a admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.086309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1e08-c248-49fd-ac99-e32533afa81a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.086780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1e08-c248-49fd-ac99-e32533afa81a admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.087023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6eedfb-2876-4d92-9db8-9ce9fec992b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef89791b177f4bf0ba916f133c79f33a, user_id=20fd230cc30f4980bedebc75c620d707, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.088057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b1e08-c248-49fd-ac99-e32533afa81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.091005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b1e08-c248-49fd-ac99-e32533afa81a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.091995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6eedfb-2876-4d92-9db8-9ce9fec992b3 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.112878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 802/1601] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:43.112878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aae1fa-4158-42d7-a170-911f5e100a0c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.112878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aae1fa-4158-42d7-a170-911f5e100a0c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.112878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aae1fa-4158-42d7-a170-911f5e100a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.125505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 800/1602] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:43 2022] PUT /identity/v3/projects/ef89791b177f4bf0ba916f133c79f33a/users/20fd230cc30f4980bedebc75c620d707/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:43.125505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6eedfb-2876-4d92-9db8-9ce9fec992b3 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.135984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-18aae1fa-4158-42d7-a170-911f5e100a0c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:43.137334 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-18aae1fa-4158-42d7-a170-911f5e100a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:43.213289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6eedfb-2876-4d92-9db8-9ce9fec992b3 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.220630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cf0b3-5d2a-4f04-9be7-48584bffa681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.221890 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cf0b3-5d2a-4f04-9be7-48584bffa681 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.222136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cf0b3-5d2a-4f04-9be7-48584bffa681 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.222419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cf0b3-5d2a-4f04-9be7-48584bffa681 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.223225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cf0b3-5d2a-4f04-9be7-48584bffa681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.225003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cf0b3-5d2a-4f04-9be7-48584bffa681 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.232297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 803/1603] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:43.239160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b7c3c-4bf8-4170-820a-3eaf1df2b704 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.239499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b7c3c-4bf8-4170-820a-3eaf1df2b704 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.239929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b7c3c-4bf8-4170-820a-3eaf1df2b704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.246262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 801/1604] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:43.246262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cf0b3-5d2a-4f04-9be7-48584bffa681 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.257502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cf0b3-5d2a-4f04-9be7-48584bffa681 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.264050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de16ad-6128-44dc-81c9-87a54a0da423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.264888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de16ad-6128-44dc-81c9-87a54a0da423 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.265240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-473b7c3c-4bf8-4170-820a-3eaf1df2b704 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:43.265431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de16ad-6128-44dc-81c9-87a54a0da423 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.265682 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de16ad-6128-44dc-81c9-87a54a0da423 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.266520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de16ad-6128-44dc-81c9-87a54a0da423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.266886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-473b7c3c-4bf8-4170-820a-3eaf1df2b704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:43.268983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de16ad-6128-44dc-81c9-87a54a0da423 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.276607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 804/1605] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:43.277526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b7c3c-4bf8-4170-820a-3eaf1df2b704 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.280653 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 802/1606] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:43.281874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de16ad-6128-44dc-81c9-87a54a0da423 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.293792 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b7c3c-4bf8-4170-820a-3eaf1df2b704 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.302063 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f2455-199d-41f0-a9e0-ce5e64febb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.303718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f2455-199d-41f0-a9e0-ce5e64febb83 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.304186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f2455-199d-41f0-a9e0-ce5e64febb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.304722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f2455-199d-41f0-a9e0-ce5e64febb83 admin admin] PATH_INFO: `/v3/projects/a527595604ba49a58fd0ac409a8d2066/users/1c0e1d5bf55345d3b9dd61480235ec21/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.336274 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f2455-199d-41f0-a9e0-ce5e64febb83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a527595604ba49a58fd0ac409a8d2066, user_id=1c0e1d5bf55345d3b9dd61480235ec21, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.339656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f2455-199d-41f0-a9e0-ce5e64febb83 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.354226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de16ad-6128-44dc-81c9-87a54a0da423 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.360140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 805/1607] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:43 2022] PUT /identity/v3/projects/a527595604ba49a58fd0ac409a8d2066/users/1c0e1d5bf55345d3b9dd61480235ec21/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:43.364445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f2455-199d-41f0-a9e0-ce5e64febb83 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.410966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9d58e-1c17-4aa3-9ebd-399bf8d33757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.411549 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9d58e-1c17-4aa3-9ebd-399bf8d33757 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.412019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9d58e-1c17-4aa3-9ebd-399bf8d33757 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.412481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9d58e-1c17-4aa3-9ebd-399bf8d33757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.429156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f2455-199d-41f0-a9e0-ce5e64febb83 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.434772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60314c68-fc06-49b2-b157-90d2d5f65591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.435589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60314c68-fc06-49b2-b157-90d2d5f65591 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.436035 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60314c68-fc06-49b2-b157-90d2d5f65591 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.436442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60314c68-fc06-49b2-b157-90d2d5f65591 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.437309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60314c68-fc06-49b2-b157-90d2d5f65591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.439179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60314c68-fc06-49b2-b157-90d2d5f65591 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.460312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 806/1608] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:43.460312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54830-329d-43ef-afe3-8118f071d647 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.460312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54830-329d-43ef-afe3-8118f071d647 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.460312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54830-329d-43ef-afe3-8118f071d647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.482357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b2a54830-329d-43ef-afe3-8118f071d647 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:43.484293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b2a54830-329d-43ef-afe3-8118f071d647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:43.489252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9d58e-1c17-4aa3-9ebd-399bf8d33757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.491872 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9d58e-1c17-4aa3-9ebd-399bf8d33757 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.496466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 807/1609] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:43.497810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a54830-329d-43ef-afe3-8118f071d647 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.508729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 803/1610] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:43.510192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9d58e-1c17-4aa3-9ebd-399bf8d33757 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.512044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a54830-329d-43ef-afe3-8118f071d647 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.517237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5ef67-f52d-4c8c-b90c-5adf57610ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.518695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5ef67-f52d-4c8c-b90c-5adf57610ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.519014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5ef67-f52d-4c8c-b90c-5adf57610ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.519382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5ef67-f52d-4c8c-b90c-5adf57610ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.527557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9d58e-1c17-4aa3-9ebd-399bf8d33757 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.530420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5ef67-f52d-4c8c-b90c-5adf57610ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.531872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5ef67-f52d-4c8c-b90c-5adf57610ea1 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.536026 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bad861-5ef4-4f41-9b97-dfbfcb08773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.537762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bad861-5ef4-4f41-9b97-dfbfcb08773e admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.538402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bad861-5ef4-4f41-9b97-dfbfcb08773e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.539063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bad861-5ef4-4f41-9b97-dfbfcb08773e admin admin] PATH_INFO: `/v3/projects/203eb74b5a2c444aac67940d775ecfc3/users/8b2980196e0f4fafa5131dd5de02ac75/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.542312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 808/1611] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:43.543266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5ef67-f52d-4c8c-b90c-5adf57610ea1 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.554120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5ef67-f52d-4c8c-b90c-5adf57610ea1 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.559101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bcbcc-b400-459a-bbe1-49fefd9ea07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.560359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bcbcc-b400-459a-bbe1-49fefd9ea07c admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.560640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bcbcc-b400-459a-bbe1-49fefd9ea07c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.560904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bcbcc-b400-459a-bbe1-49fefd9ea07c admin admin] PATH_INFO: `/v3/projects/d42ea16d0ee64f398b01b206c1960b08/users/0622aea8005b4b82bcb921089974775d/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.572262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bad861-5ef4-4f41-9b97-dfbfcb08773e admin admin] RBAC: Authorizing `identity:create_grant(project_id=203eb74b5a2c444aac67940d775ecfc3, user_id=8b2980196e0f4fafa5131dd5de02ac75, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.577006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bad861-5ef4-4f41-9b97-dfbfcb08773e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.587345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bcbcc-b400-459a-bbe1-49fefd9ea07c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42ea16d0ee64f398b01b206c1960b08, user_id=0622aea8005b4b82bcb921089974775d, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.593435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bcbcc-b400-459a-bbe1-49fefd9ea07c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.600111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 804/1612] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:43 2022] PUT /identity/v3/projects/203eb74b5a2c444aac67940d775ecfc3/users/8b2980196e0f4fafa5131dd5de02ac75/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:43.600111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bad861-5ef4-4f41-9b97-dfbfcb08773e admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.630317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 809/1613] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:43 2022] PUT /identity/v3/projects/d42ea16d0ee64f398b01b206c1960b08/users/0622aea8005b4b82bcb921089974775d/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:43.630317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bcbcc-b400-459a-bbe1-49fefd9ea07c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.670641 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bad861-5ef4-4f41-9b97-dfbfcb08773e admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.729998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8c225-27c1-4c6d-b211-b140268c4237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.730841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8c225-27c1-4c6d-b211-b140268c4237 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.731124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8c225-27c1-4c6d-b211-b140268c4237 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.731398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8c225-27c1-4c6d-b211-b140268c4237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.744250 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8c225-27c1-4c6d-b211-b140268c4237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.745852 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8c225-27c1-4c6d-b211-b140268c4237 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.747824 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bcbcc-b400-459a-bbe1-49fefd9ea07c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.756952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f054a3-b41a-4ee2-8dd6-fe7d5d52b5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.758252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f054a3-b41a-4ee2-8dd6-fe7d5d52b5ff admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.758909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f054a3-b41a-4ee2-8dd6-fe7d5d52b5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.759537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f054a3-b41a-4ee2-8dd6-fe7d5d52b5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.760831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 805/1614] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:43.761179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f054a3-b41a-4ee2-8dd6-fe7d5d52b5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.761804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8c225-27c1-4c6d-b211-b140268c4237 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.764731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f054a3-b41a-4ee2-8dd6-fe7d5d52b5ff admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.788725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 810/1615] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:43.790594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f054a3-b41a-4ee2-8dd6-fe7d5d52b5ff admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.798085 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8c225-27c1-4c6d-b211-b140268c4237 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.804850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da7a3e-baef-4ca6-9bec-d3b4e25f0842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.805806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da7a3e-baef-4ca6-9bec-d3b4e25f0842 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.806083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da7a3e-baef-4ca6-9bec-d3b4e25f0842 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.806501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da7a3e-baef-4ca6-9bec-d3b4e25f0842 admin admin] PATH_INFO: `/v3/projects/de17783e3b08413a88dd343eb4c4f28a/users/4e94408aeb5d4f809419b3c7865224b2/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.812429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f054a3-b41a-4ee2-8dd6-fe7d5d52b5ff admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.828722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da7a3e-baef-4ca6-9bec-d3b4e25f0842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de17783e3b08413a88dd343eb4c4f28a, user_id=4e94408aeb5d4f809419b3c7865224b2, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.829144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068e89e-c576-482f-8cc8-f7aa9ca6b8bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:43.830035 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068e89e-c576-482f-8cc8-f7aa9ca6b8bc service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.830331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068e89e-c576-482f-8cc8-f7aa9ca6b8bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.831271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068e89e-c576-482f-8cc8-f7aa9ca6b8bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.833510 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da7a3e-baef-4ca6-9bec-d3b4e25f0842 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.845592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068e89e-c576-482f-8cc8-f7aa9ca6b8bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:43.847466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068e89e-c576-482f-8cc8-f7aa9ca6b8bc service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:43.855599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 806/1616] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:43 2022] PUT /identity/v3/projects/de17783e3b08413a88dd343eb4c4f28a/users/4e94408aeb5d4f809419b3c7865224b2/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:43.857438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e5aa6-b789-4073-84d4-ff0df2001a78 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.857724 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e5aa6-b789-4073-84d4-ff0df2001a78 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.858048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e5aa6-b789-4073-84d4-ff0df2001a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.873027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 811/1617] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:43.874772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f5a6c-41e3-4164-a686-6043a80e0192 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:43.875745 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f5a6c-41e3-4164-a686-6043a80e0192 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:43.876309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f5a6c-41e3-4164-a686-6043a80e0192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:43.891960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-691e5aa6-b789-4073-84d4-ff0df2001a78 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:43.893389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-691e5aa6-b789-4073-84d4-ff0df2001a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:43.902159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-118f5a6c-41e3-4164-a686-6043a80e0192 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:43.903676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-118f5a6c-41e3-4164-a686-6043a80e0192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:43.984261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 807/1618] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:43.984979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e5aa6-b789-4073-84d4-ff0df2001a78 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:43.997433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e5aa6-b789-4073-84d4-ff0df2001a78 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:43.999378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 812/1619] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:44.001761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace2889-b7e3-4831-91a4-da948a5f6a9b None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:44.003147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace2889-b7e3-4831-91a4-da948a5f6a9b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:44.003566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace2889-b7e3-4831-91a4-da948a5f6a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:44.007765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05fc67-ad73-4606-bc29-c37ebd50a284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:44.008990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05fc67-ad73-4606-bc29-c37ebd50a284 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:44.009365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05fc67-ad73-4606-bc29-c37ebd50a284 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:44.009760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05fc67-ad73-4606-bc29-c37ebd50a284 admin admin] PATH_INFO: `/v3/projects/a527595604ba49a58fd0ac409a8d2066/users/1c0e1d5bf55345d3b9dd61480235ec21/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:44.013717 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05fc67-ad73-4606-bc29-c37ebd50a284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a527595604ba49a58fd0ac409a8d2066, user_id=1c0e1d5bf55345d3b9dd61480235ec21, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:44.016766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05fc67-ad73-4606-bc29-c37ebd50a284 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:44.031902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 808/1620] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:43 2022] PUT /identity/v3/projects/a527595604ba49a58fd0ac409a8d2066/users/1c0e1d5bf55345d3b9dd61480235ec21/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:44.032610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05fc67-ad73-4606-bc29-c37ebd50a284 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:44.053112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-eace2889-b7e3-4831-91a4-da948a5f6a9b None None] MFA Rules not processed for user `4e94408aeb5d4f809419b3c7865224b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:44.055451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-eace2889-b7e3-4831-91a4-da948a5f6a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:44.117538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05fc67-ad73-4606-bc29-c37ebd50a284 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:44.122174 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c72dc-1787-4b78-b351-b0596cd1475f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:44.123179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c72dc-1787-4b78-b351-b0596cd1475f service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:44.123585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c72dc-1787-4b78-b351-b0596cd1475f service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:44.124058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c72dc-1787-4b78-b351-b0596cd1475f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:44.188002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 813/1621] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:44.194391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d5104-6827-4a40-bd0f-1bd43d5331e2 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:44.194837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d5104-6827-4a40-bd0f-1bd43d5331e2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:44.195339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d5104-6827-4a40-bd0f-1bd43d5331e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:44.223143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c72dc-1787-4b78-b351-b0596cd1475f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:44.225670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c72dc-1787-4b78-b351-b0596cd1475f service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:44.234372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-423d5104-6827-4a40-bd0f-1bd43d5331e2 None None] MFA Rules not processed for user `1c0e1d5bf55345d3b9dd61480235ec21`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:44.236077 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-423d5104-6827-4a40-bd0f-1bd43d5331e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:44.239046 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 809/1622] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:44.240660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c72dc-1787-4b78-b351-b0596cd1475f service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:44.251378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c72dc-1787-4b78-b351-b0596cd1475f service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:44.256731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9132e-7d82-453b-ad30-3fba99f629b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:44.257478 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9132e-7d82-453b-ad30-3fba99f629b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:44.258251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9132e-7d82-453b-ad30-3fba99f629b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:44.258661 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9132e-7d82-453b-ad30-3fba99f629b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:44.273572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9132e-7d82-453b-ad30-3fba99f629b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:44.275303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9132e-7d82-453b-ad30-3fba99f629b2 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:44.290251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 810/1623] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:44.292148 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea1785-f884-4cfa-86f8-d10e37eb47e8 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:44.292555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea1785-f884-4cfa-86f8-d10e37eb47e8 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:44.292942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea1785-f884-4cfa-86f8-d10e37eb47e8 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:44.294558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 811/1624] 10.222.0.79 () {54 vars in 961 bytes} [Fri May 13 13:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:55:44.302453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee7df0-ca7c-43fa-8973-72be35f76c2c None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:44.302453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee7df0-ca7c-43fa-8973-72be35f76c2c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:44.302954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee7df0-ca7c-43fa-8973-72be35f76c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:44.328371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-37ee7df0-ca7c-43fa-8973-72be35f76c2c None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:44.329797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-37ee7df0-ca7c-43fa-8973-72be35f76c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:44.411948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 814/1625] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:44.429520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d5104-6827-4a40-bd0f-1bd43d5331e2 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:44.441950 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 812/1626] 10.222.0.79 () {58 vars in 1058 bytes} [Fri May 13 13:55:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:44.451066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee7df0-ca7c-43fa-8973-72be35f76c2c None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:44.495915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee7df0-ca7c-43fa-8973-72be35f76c2c None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:44.526274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce7652-06df-42f5-95e7-0aebae68bf06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:44.530084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce7652-06df-42f5-95e7-0aebae68bf06 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:44.530924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce7652-06df-42f5-95e7-0aebae68bf06 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:44.531691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce7652-06df-42f5-95e7-0aebae68bf06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:44.556405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce7652-06df-42f5-95e7-0aebae68bf06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:44.558504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d5104-6827-4a40-bd0f-1bd43d5331e2 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:44.559830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce7652-06df-42f5-95e7-0aebae68bf06 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:44.566701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfee1f0-4837-4002-a937-c9144c13b6dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:44.568060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfee1f0-4837-4002-a937-c9144c13b6dc service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:44.568704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfee1f0-4837-4002-a937-c9144c13b6dc service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:44.569550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfee1f0-4837-4002-a937-c9144c13b6dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:44.582501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 813/1627] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:44.588498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfee1f0-4837-4002-a937-c9144c13b6dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:44.591817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfee1f0-4837-4002-a937-c9144c13b6dc service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:44.624125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 815/1628] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:55:45.679971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce7652-06df-42f5-95e7-0aebae68bf06 service nova] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:45.718781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce7652-06df-42f5-95e7-0aebae68bf06 service nova] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:45.726509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:45.728100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:45.728725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:45.729368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] PATH_INFO: `/v3/users/da08dcccc94842ba9911450666c4fa72` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:45.731334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=da08dcccc94842ba9911450666c4fa72)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:45.732680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: WARNING py.warnings [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:55:45.732680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: warnings.warn(msg) May 13 13:55:45.732680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: May 13 13:55:45.734351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:45.799889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adc0de-0c7a-48d7-8d30-c27cb622ed37 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:45.799889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adc0de-0c7a-48d7-8d30-c27cb622ed37 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:45.800530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adc0de-0c7a-48d7-8d30-c27cb622ed37 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:45.801854 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 816/1629] 10.222.0.79 () {54 vars in 961 bytes} [Fri May 13 13:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:55:45.808711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e06c76-b397-492e-a88a-94879c9d5f09 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:45.809105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e06c76-b397-492e-a88a-94879c9d5f09 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:45.809556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e06c76-b397-492e-a88a-94879c9d5f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:45.841632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-50e06c76-b397-492e-a88a-94879c9d5f09 None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:45.843017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-50e06c76-b397-492e-a88a-94879c9d5f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:45.850416 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da08dcccc94842ba9911450666c4fa72'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:45.851959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da08dcccc94842ba9911450666c4fa72'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:45.867368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 817/1630] 10.222.0.79 () {58 vars in 1058 bytes} [Fri May 13 13:55:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:45.878895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957af69-d82b-4ecf-a935-9ff0552903b9 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:45.879199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957af69-d82b-4ecf-a935-9ff0552903b9 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:45.879504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957af69-d82b-4ecf-a935-9ff0552903b9 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:45.881316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 818/1631] 10.222.0.79 () {54 vars in 912 bytes} [Fri May 13 13:55:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:55:45.893265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da08dcccc94842ba9911450666c4fa72'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:45.893888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f49096-978b-4f3e-8551-504470acb0b3 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:45.894179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f49096-978b-4f3e-8551-504470acb0b3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:45.894338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da08dcccc94842ba9911450666c4fa72'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:45.894497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f49096-978b-4f3e-8551-504470acb0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:45.910631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5de3e98b-c00f-44cb-954d-ee06d1469f6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da08dcccc94842ba9911450666c4fa72'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:45.931057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-27f49096-978b-4f3e-8551-504470acb0b3 None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:45.932485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-27f49096-978b-4f3e-8551-504470acb0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:45.935622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 814/1632] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:55:45 2022] DELETE /identity/v3/users/da08dcccc94842ba9911450666c4fa72 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:45.937375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ee2cb-c520-4bb6-a297-febebe43ee58 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:45.937677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ee2cb-c520-4bb6-a297-febebe43ee58 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:45.938020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ee2cb-c520-4bb6-a297-febebe43ee58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:45.965995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-8b3ee2cb-c520-4bb6-a297-febebe43ee58 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:45.967680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8b3ee2cb-c520-4bb6-a297-febebe43ee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:46.083424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 819/1633] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 13:55:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:46.104928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f49096-978b-4f3e-8551-504470acb0b3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:46.106669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-27f49096-978b-4f3e-8551-504470acb0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:46.141883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 815/1634] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:46.179946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ee2cb-c520-4bb6-a297-febebe43ee58 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:46.182515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8b3ee2cb-c520-4bb6-a297-febebe43ee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:46.286515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f49096-978b-4f3e-8551-504470acb0b3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:46.292091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a159373-6ae9-415f-86fe-3121768eb5f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:46.293132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a159373-6ae9-415f-86fe-3121768eb5f6 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:46.293565 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a159373-6ae9-415f-86fe-3121768eb5f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:46.293897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a159373-6ae9-415f-86fe-3121768eb5f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:46.308148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a159373-6ae9-415f-86fe-3121768eb5f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:46.310845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a159373-6ae9-415f-86fe-3121768eb5f6 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:46.325595 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 820/1635] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:55:46.369742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ec1df-02fc-4503-8fbb-38d44ce7ef4f None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:46.370401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ec1df-02fc-4503-8fbb-38d44ce7ef4f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:46.372244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ec1df-02fc-4503-8fbb-38d44ce7ef4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:46.394397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ee2cb-c520-4bb6-a297-febebe43ee58 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:46.399636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e029da-87ca-4b13-938f-72a4b9f0b785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:46.400505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e029da-87ca-4b13-938f-72a4b9f0b785 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:46.401120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e029da-87ca-4b13-938f-72a4b9f0b785 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:46.401631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e029da-87ca-4b13-938f-72a4b9f0b785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:46.418602 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e029da-87ca-4b13-938f-72a4b9f0b785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:46.424079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e029da-87ca-4b13-938f-72a4b9f0b785 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:46.450853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 816/1636] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:55:46.471380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6b3ec1df-02fc-4503-8fbb-38d44ce7ef4f None None] MFA Rules not processed for user `c672faa2a3d849adb0898cb11b5e5038`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:46.476069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6b3ec1df-02fc-4503-8fbb-38d44ce7ef4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:46.642532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c74c5-3d65-47f9-bb03-6d900b381d51 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:46.642752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c74c5-3d65-47f9-bb03-6d900b381d51 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:46.643076 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c74c5-3d65-47f9-bb03-6d900b381d51 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:46.645621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 817/1637] 10.222.0.79 () {54 vars in 912 bytes} [Fri May 13 13:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:55:46.657288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec08e4-1ccf-4123-84bd-25215738a140 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:46.657288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec08e4-1ccf-4123-84bd-25215738a140 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:46.657288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec08e4-1ccf-4123-84bd-25215738a140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:46.699795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-ecec08e4-1ccf-4123-84bd-25215738a140 None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:46.701122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ecec08e4-1ccf-4123-84bd-25215738a140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:46.721342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 821/1638] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:46 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:55:46.728407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ec1df-02fc-4503-8fbb-38d44ce7ef4f None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:46.729619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 818/1639] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 13:55:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:46.742917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6b3ec1df-02fc-4503-8fbb-38d44ce7ef4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:46.749862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec08e4-1ccf-4123-84bd-25215738a140 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:46.751415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ecec08e4-1ccf-4123-84bd-25215738a140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:46.768699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ec1df-02fc-4503-8fbb-38d44ce7ef4f None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:46.773756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec08e4-1ccf-4123-84bd-25215738a140 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:46.778451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb583c-1f64-4125-96a4-9902f492157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:46.779636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb583c-1f64-4125-96a4-9902f492157d admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:46.780028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb583c-1f64-4125-96a4-9902f492157d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:46.780549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb583c-1f64-4125-96a4-9902f492157d admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:46.781678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb583c-1f64-4125-96a4-9902f492157d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:46.785215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb583c-1f64-4125-96a4-9902f492157d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:46.789536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe325aa-d75d-4345-9de4-e2a55cb47458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:46.790820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe325aa-d75d-4345-9de4-e2a55cb47458 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:46.791220 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe325aa-d75d-4345-9de4-e2a55cb47458 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:46.791508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe325aa-d75d-4345-9de4-e2a55cb47458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:46.821121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe325aa-d75d-4345-9de4-e2a55cb47458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:46.824588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe325aa-d75d-4345-9de4-e2a55cb47458 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:46.856002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 819/1640] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:55:46.865195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 822/1641] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:46 2022] POST /identity/v3/projects => generated 403 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:46.866218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe325aa-d75d-4345-9de4-e2a55cb47458 service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:46.878698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe325aa-d75d-4345-9de4-e2a55cb47458 service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:46.967369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f8f1a-2d52-4cfb-bcc5-e826a5c485c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:46.969043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f8f1a-2d52-4cfb-bcc5-e826a5c485c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:46.969372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f8f1a-2d52-4cfb-bcc5-e826a5c485c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:46.969637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f8f1a-2d52-4cfb-bcc5-e826a5c485c8 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:46.970334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f8f1a-2d52-4cfb-bcc5-e826a5c485c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:46.973765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f8f1a-2d52-4cfb-bcc5-e826a5c485c8 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:47.010453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 820/1642] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:46 2022] POST /identity/v3/users => generated 308 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:55:47.015932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb583c-1f64-4125-96a4-9902f492157d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:47.037606 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb583c-1f64-4125-96a4-9902f492157d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:47.045629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947b2f5-6e4a-43bd-a3d9-df74189f7370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:47.046439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947b2f5-6e4a-43bd-a3d9-df74189f7370 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:47.046863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947b2f5-6e4a-43bd-a3d9-df74189f7370 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:47.047173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947b2f5-6e4a-43bd-a3d9-df74189f7370 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:47.048062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947b2f5-6e4a-43bd-a3d9-df74189f7370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:47.052359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947b2f5-6e4a-43bd-a3d9-df74189f7370 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:47.095237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 823/1643] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:47.102466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f8f1a-2d52-4cfb-bcc5-e826a5c485c8 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:47.124256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f8f1a-2d52-4cfb-bcc5-e826a5c485c8 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:47.137561 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cb456-3418-4aa0-adad-875a2e182c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:47.138483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cb456-3418-4aa0-adad-875a2e182c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:47.138838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cb456-3418-4aa0-adad-875a2e182c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:47.139126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cb456-3418-4aa0-adad-875a2e182c15 admin admin] PATH_INFO: `/v3/projects/43e71917c48246fab1030a732f2616ee/users/fff87b1f656a4d3d8559d013711b0e11/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:47.179337 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cb456-3418-4aa0-adad-875a2e182c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e71917c48246fab1030a732f2616ee, user_id=fff87b1f656a4d3d8559d013711b0e11, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:47.187952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cb456-3418-4aa0-adad-875a2e182c15 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:47.218614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 821/1644] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:47 2022] PUT /identity/v3/projects/43e71917c48246fab1030a732f2616ee/users/fff87b1f656a4d3d8559d013711b0e11/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:47.225148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947b2f5-6e4a-43bd-a3d9-df74189f7370 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:47.243972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947b2f5-6e4a-43bd-a3d9-df74189f7370 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:47.409163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060c5fc-0258-44e5-bd48-0f9910e3d4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:47.409910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c5fc-0258-44e5-bd48-0f9910e3d4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:47.410252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c5fc-0258-44e5-bd48-0f9910e3d4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:47.410538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c5fc-0258-44e5-bd48-0f9910e3d4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:47.413144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060c5fc-0258-44e5-bd48-0f9910e3d4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:47.415591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060c5fc-0258-44e5-bd48-0f9910e3d4c8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:47.428767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 824/1645] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:47.441215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cb456-3418-4aa0-adad-875a2e182c15 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:47.485528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cb456-3418-4aa0-adad-875a2e182c15 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:47.499917 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3dfe2-b58a-46d7-bbf9-402c56a1913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:47.503314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3dfe2-b58a-46d7-bbf9-402c56a1913e admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:47.504827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3dfe2-b58a-46d7-bbf9-402c56a1913e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:47.505321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3dfe2-b58a-46d7-bbf9-402c56a1913e admin admin] PATH_INFO: `/v3/projects/43e71917c48246fab1030a732f2616ee/users/fff87b1f656a4d3d8559d013711b0e11/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:47.508146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3dfe2-b58a-46d7-bbf9-402c56a1913e admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e71917c48246fab1030a732f2616ee, user_id=fff87b1f656a4d3d8559d013711b0e11, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:47.511695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3dfe2-b58a-46d7-bbf9-402c56a1913e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:47.528612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 822/1646] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:47 2022] PUT /identity/v3/projects/43e71917c48246fab1030a732f2616ee/users/fff87b1f656a4d3d8559d013711b0e11/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:47.552789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060c5fc-0258-44e5-bd48-0f9910e3d4c8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:47.556059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6060c5fc-0258-44e5-bd48-0f9910e3d4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:47.576724 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060c5fc-0258-44e5-bd48-0f9910e3d4c8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:47.734906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:47.735743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:47.736225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:47.736581 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] PATH_INFO: `/v3/users/4e94408aeb5d4f809419b3c7865224b2` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:47.739294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e94408aeb5d4f809419b3c7865224b2)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:47.743600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:47.861621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e94408aeb5d4f809419b3c7865224b2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:47.893086 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e94408aeb5d4f809419b3c7865224b2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:47.921400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e94408aeb5d4f809419b3c7865224b2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:47.922839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e94408aeb5d4f809419b3c7865224b2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:47.941553 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e94408aeb5d4f809419b3c7865224b2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:47.958341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 825/1647] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:55:47 2022] DELETE /identity/v3/users/4e94408aeb5d4f809419b3c7865224b2 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:47.961981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e74ec-76be-462c-898a-5864c87f7452 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:47.962895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e74ec-76be-462c-898a-5864c87f7452 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:47.962895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e74ec-76be-462c-898a-5864c87f7452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:48.006296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-309e74ec-76be-462c-898a-5864c87f7452 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:48.008236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-309e74ec-76be-462c-898a-5864c87f7452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:48.213238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 823/1648] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:48.293630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:48.296822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:48.407459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c6c6c-598e-406f-acd0-9c04164be69d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:48.413698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286570b9-cf5c-4cc7-ba47-18e98088925e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:48.414747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286570b9-cf5c-4cc7-ba47-18e98088925e service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:48.415290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286570b9-cf5c-4cc7-ba47-18e98088925e service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:48.415762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286570b9-cf5c-4cc7-ba47-18e98088925e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:48.435980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286570b9-cf5c-4cc7-ba47-18e98088925e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:48.437883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286570b9-cf5c-4cc7-ba47-18e98088925e service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:48.457963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 826/1649] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:48.496975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e74ec-76be-462c-898a-5864c87f7452 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:48.498260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-309e74ec-76be-462c-898a-5864c87f7452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:48.523689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e74ec-76be-462c-898a-5864c87f7452 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:48.540023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:48.541665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:48.543538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:48.544099 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] PATH_INFO: `/v3/projects/0887bb8970724681908617358b2e9831` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:48.546517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0887bb8970724681908617358b2e9831)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:48.556081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:48.709731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0887bb8970724681908617358b2e9831', 'request_id': 'req-497e36b7-fd7e-449b-97b0-aeca039d04e4'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:48.735659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0887bb8970724681908617358b2e9831', 'request_id': 'req-497e36b7-fd7e-449b-97b0-aeca039d04e4'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:48.750861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0887bb8970724681908617358b2e9831', 'request_id': 'req-497e36b7-fd7e-449b-97b0-aeca039d04e4'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:48.774607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] The token cache is being invalidate because project 0887bb8970724681908617358b2e9831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:55:48.776098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:48.777040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-497e36b7-fd7e-449b-97b0-aeca039d04e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:48.785430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 824/1650] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:55:48 2022] DELETE /identity/v3/projects/0887bb8970724681908617358b2e9831 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:48.785744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286570b9-cf5c-4cc7-ba47-18e98088925e service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:48.802308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286570b9-cf5c-4cc7-ba47-18e98088925e service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:48.947535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:48.949407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:48.949935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:48.950374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] PATH_INFO: `/v3/users/1c0e1d5bf55345d3b9dd61480235ec21` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:48.953319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c0e1d5bf55345d3b9dd61480235ec21)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:48.957476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:49.057089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c0e1d5bf55345d3b9dd61480235ec21'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:49.077981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c0e1d5bf55345d3b9dd61480235ec21'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:49.095107 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c0e1d5bf55345d3b9dd61480235ec21'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:49.096939 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c0e1d5bf55345d3b9dd61480235ec21'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:49.119095 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfc9879c-b6ae-481f-a9ae-68d45e80e31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c0e1d5bf55345d3b9dd61480235ec21'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:49.131229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 827/1651] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:55:48 2022] DELETE /identity/v3/users/1c0e1d5bf55345d3b9dd61480235ec21 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:49.453779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49855221-fd26-4b81-81f1-16742202ac40 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:49.454782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49855221-fd26-4b81-81f1-16742202ac40 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:49.454782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49855221-fd26-4b81-81f1-16742202ac40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:49.544993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-49855221-fd26-4b81-81f1-16742202ac40 None None] MFA Rules not processed for user `0622aea8005b4b82bcb921089974775d`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:49.550630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-49855221-fd26-4b81-81f1-16742202ac40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:49.766612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 825/1652] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:49.772889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccb2b2-fd02-4644-ae74-083612594b0c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:49.774646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccb2b2-fd02-4644-ae74-083612594b0c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:49.775497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccb2b2-fd02-4644-ae74-083612594b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:49.824259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e7ccb2b2-fd02-4644-ae74-083612594b0c None None] MFA Rules not processed for user `0622aea8005b4b82bcb921089974775d`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:49.828358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e7ccb2b2-fd02-4644-ae74-083612594b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:49.855744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 828/1653] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:55:49.891952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49855221-fd26-4b81-81f1-16742202ac40 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:49.917915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-49855221-fd26-4b81-81f1-16742202ac40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:50.149740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49855221-fd26-4b81-81f1-16742202ac40 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:50.160252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45d02d-fb4d-4e8e-a359-f3ab2e4bbb56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:50.163709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45d02d-fb4d-4e8e-a359-f3ab2e4bbb56 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:50.164160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45d02d-fb4d-4e8e-a359-f3ab2e4bbb56 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:50.165014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45d02d-fb4d-4e8e-a359-f3ab2e4bbb56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:50.183660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45d02d-fb4d-4e8e-a359-f3ab2e4bbb56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:50.185857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45d02d-fb4d-4e8e-a359-f3ab2e4bbb56 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:50.212554 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 826/1654] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:50.882069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccb2b2-fd02-4644-ae74-083612594b0c None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:50.884763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e7ccb2b2-fd02-4644-ae74-083612594b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:51.078224 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45d02d-fb4d-4e8e-a359-f3ab2e4bbb56 service nova] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:51.084056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8c45d02d-fb4d-4e8e-a359-f3ab2e4bbb56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:51.127970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccb2b2-fd02-4644-ae74-083612594b0c None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:51.135273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45d02d-fb4d-4e8e-a359-f3ab2e4bbb56 service nova] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:51.143072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:51.144797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:51.145790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:51.145790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] PATH_INFO: `/v3/projects/a527595604ba49a58fd0ac409a8d2066` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:51.149248 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:51.151041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a527595604ba49a58fd0ac409a8d2066)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:51.151933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:51.152766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:51.153537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] PATH_INFO: `/v3/projects/de17783e3b08413a88dd343eb4c4f28a` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:51.156823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: WARNING py.warnings [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:55:51.156823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: warnings.warn(msg) May 13 13:55:51.156823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: May 13 13:55:51.158571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] RBAC: Authorizing `identity:delete_project(project_id=de17783e3b08413a88dd343eb4c4f28a)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:51.159190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:51.164412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:51.293120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de17783e3b08413a88dd343eb4c4f28a', 'request_id': 'req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:51.317727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de17783e3b08413a88dd343eb4c4f28a', 'request_id': 'req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:51.335987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de17783e3b08413a88dd343eb4c4f28a', 'request_id': 'req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:51.349950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a527595604ba49a58fd0ac409a8d2066', 'request_id': 'req-4fef34b0-af97-476b-900d-3f69107c410d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:51.362963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a527595604ba49a58fd0ac409a8d2066', 'request_id': 'req-4fef34b0-af97-476b-900d-3f69107c410d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:51.371349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] The token cache is being invalidate because project de17783e3b08413a88dd343eb4c4f28a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:55:51.372020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:51.373059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1f22ae64-c4d0-4b0d-a6d7-2b157d139fca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:51.382249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 827/1655] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:55:51 2022] DELETE /identity/v3/projects/de17783e3b08413a88dd343eb4c4f28a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:51.391830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a527595604ba49a58fd0ac409a8d2066', 'request_id': 'req-4fef34b0-af97-476b-900d-3f69107c410d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:51.393966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7aa6a-38d6-40a5-a091-605af30c7fde None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:51.394365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7aa6a-38d6-40a5-a091-605af30c7fde None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:51.394769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7aa6a-38d6-40a5-a091-605af30c7fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:51.450105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-0ee7aa6a-38d6-40a5-a091-605af30c7fde None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:51.453052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] The token cache is being invalidate because project a527595604ba49a58fd0ac409a8d2066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:55:51.456220 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0ee7aa6a-38d6-40a5-a091-605af30c7fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:51.457011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:51.460047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-4fef34b0-af97-476b-900d-3f69107c410d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:51.465473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 829/1656] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:55:50 2022] DELETE /identity/v3/projects/a527595604ba49a58fd0ac409a8d2066 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:51.475499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386adf0-7b62-4da8-997b-9d5d6894405f None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:51.476340 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386adf0-7b62-4da8-997b-9d5d6894405f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:51.477071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386adf0-7b62-4da8-997b-9d5d6894405f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:51.568652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b386adf0-7b62-4da8-997b-9d5d6894405f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:51.570488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b386adf0-7b62-4da8-997b-9d5d6894405f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:51.627876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 828/1657] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:55:51.639694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d2e30-2909-4d57-80ca-debd202f0bee None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:51.640021 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d2e30-2909-4d57-80ca-debd202f0bee None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:51.640444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d2e30-2909-4d57-80ca-debd202f0bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:51.676843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7d9d2e30-2909-4d57-80ca-debd202f0bee None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:51.679430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7d9d2e30-2909-4d57-80ca-debd202f0bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:51.699990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 829/1658] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:55:51.704853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d2e30-2909-4d57-80ca-debd202f0bee None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:51.717106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d2e30-2909-4d57-80ca-debd202f0bee None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:51.725176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5903a9f-786a-4130-a740-9b26b656a66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:51.726069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 830/1659] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:51.735765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f995220-6f93-4b24-82ff-6455b0bebab2 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:51.736314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5903a9f-786a-4130-a740-9b26b656a66f admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:51.736633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f995220-6f93-4b24-82ff-6455b0bebab2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:51.736633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f995220-6f93-4b24-82ff-6455b0bebab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:51.737245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5903a9f-786a-4130-a740-9b26b656a66f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:51.737245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5903a9f-786a-4130-a740-9b26b656a66f admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:51.738188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5903a9f-786a-4130-a740-9b26b656a66f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:51.738573 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5903a9f-786a-4130-a740-9b26b656a66f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:51.740578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5903a9f-786a-4130-a740-9b26b656a66f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:51.763113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 830/1660] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:55:51.769647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5903a9f-786a-4130-a740-9b26b656a66f admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:51.770964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-7f995220-6f93-4b24-82ff-6455b0bebab2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:51.772544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7f995220-6f93-4b24-82ff-6455b0bebab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:51.784447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5903a9f-786a-4130-a740-9b26b656a66f admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:51.791288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8736c87-79fa-479b-b9ad-32ace91f1963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:51.791981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736c87-79fa-479b-b9ad-32ace91f1963 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:51.792265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736c87-79fa-479b-b9ad-32ace91f1963 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:51.792545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736c87-79fa-479b-b9ad-32ace91f1963 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:51.794298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8736c87-79fa-479b-b9ad-32ace91f1963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:51.797302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8736c87-79fa-479b-b9ad-32ace91f1963 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:51.801197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 831/1661] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:55:51.805698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f995220-6f93-4b24-82ff-6455b0bebab2 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:51.820670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f995220-6f93-4b24-82ff-6455b0bebab2 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:51.840130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce4ba6-ccad-4399-afc8-f09cfeef5184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:51.840904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4ba6-ccad-4399-afc8-f09cfeef5184 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:51.841193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4ba6-ccad-4399-afc8-f09cfeef5184 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:51.841471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4ba6-ccad-4399-afc8-f09cfeef5184 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:51.842223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce4ba6-ccad-4399-afc8-f09cfeef5184 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:51.842535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce4ba6-ccad-4399-afc8-f09cfeef5184 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:51.844272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce4ba6-ccad-4399-afc8-f09cfeef5184 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:51.855891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 831/1662] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:51 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:55:51.861548 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8736c87-79fa-479b-b9ad-32ace91f1963 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:51.882329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 832/1663] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:55:51.883509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce4ba6-ccad-4399-afc8-f09cfeef5184 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:51.904825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce4ba6-ccad-4399-afc8-f09cfeef5184 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:51.990203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8736c87-79fa-479b-b9ad-32ace91f1963 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:51.998042 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dcab0-c873-4ca5-b7dd-8f7605c621df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:52.000386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dcab0-c873-4ca5-b7dd-8f7605c621df admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.000944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dcab0-c873-4ca5-b7dd-8f7605c621df admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.001609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dcab0-c873-4ca5-b7dd-8f7605c621df admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.002963 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dcab0-c873-4ca5-b7dd-8f7605c621df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:52.006130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dcab0-c873-4ca5-b7dd-8f7605c621df admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:52.032019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261b4f-ee80-4def-9912-db331ac50a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:52.035453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261b4f-ee80-4def-9912-db331ac50a7e admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.036664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261b4f-ee80-4def-9912-db331ac50a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.037212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261b4f-ee80-4def-9912-db331ac50a7e admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.038319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc261b4f-ee80-4def-9912-db331ac50a7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:52.040673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc261b4f-ee80-4def-9912-db331ac50a7e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:52.063619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 832/1664] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:51 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:52.068771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dcab0-c873-4ca5-b7dd-8f7605c621df admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:52.089121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dcab0-c873-4ca5-b7dd-8f7605c621df admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:52.095832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 833/1665] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:51 2022] POST /identity/v3/projects => generated 379 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:55:52.101272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261b4f-ee80-4def-9912-db331ac50a7e admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:52.223242 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261b4f-ee80-4def-9912-db331ac50a7e admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:52.232805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17963c06-de8e-4a90-9ebe-47ef408964c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:52.234367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17963c06-de8e-4a90-9ebe-47ef408964c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.235112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17963c06-de8e-4a90-9ebe-47ef408964c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.235599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17963c06-de8e-4a90-9ebe-47ef408964c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.237722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17963c06-de8e-4a90-9ebe-47ef408964c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:52.239893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333df67-e15a-4045-9fad-f35dc9ff33a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:52.240519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17963c06-de8e-4a90-9ebe-47ef408964c3 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:52.240774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333df67-e15a-4045-9fad-f35dc9ff33a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.241072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333df67-e15a-4045-9fad-f35dc9ff33a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.241352 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333df67-e15a-4045-9fad-f35dc9ff33a0 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.242062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333df67-e15a-4045-9fad-f35dc9ff33a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:52.244034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333df67-e15a-4045-9fad-f35dc9ff33a0 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:52.264470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 833/1666] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:55:52.269330 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17963c06-de8e-4a90-9ebe-47ef408964c3 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:52.299343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 834/1667] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:52 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:52.303255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333df67-e15a-4045-9fad-f35dc9ff33a0 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:52.313357 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17963c06-de8e-4a90-9ebe-47ef408964c3 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:52.324722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333df67-e15a-4045-9fad-f35dc9ff33a0 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:52.332958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14d2d5-d98c-418b-b101-bcd512fdb422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:52.334442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14d2d5-d98c-418b-b101-bcd512fdb422 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.334639 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14d2d5-d98c-418b-b101-bcd512fdb422 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.335032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14d2d5-d98c-418b-b101-bcd512fdb422 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.335776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14d2d5-d98c-418b-b101-bcd512fdb422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:52.336605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f7f30-f85a-44b3-848f-cc6932c665ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:52.337299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f7f30-f85a-44b3-848f-cc6932c665ce admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.337576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f7f30-f85a-44b3-848f-cc6932c665ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.337850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f7f30-f85a-44b3-848f-cc6932c665ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.339625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f7f30-f85a-44b3-848f-cc6932c665ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:52.340435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14d2d5-d98c-418b-b101-bcd512fdb422 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:52.348809 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f7f30-f85a-44b3-848f-cc6932c665ce admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:52.356483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 835/1668] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:52.357630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14d2d5-d98c-418b-b101-bcd512fdb422 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:52.378072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 834/1669] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:55:52.382522 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f7f30-f85a-44b3-848f-cc6932c665ce admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:52.386256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14d2d5-d98c-418b-b101-bcd512fdb422 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:52.394525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f7f30-f85a-44b3-848f-cc6932c665ce admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:52.398207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db82f4-e356-4b7a-bffc-1c21895482ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:52.400230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db82f4-e356-4b7a-bffc-1c21895482ec admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.400354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db82f4-e356-4b7a-bffc-1c21895482ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.400779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db82f4-e356-4b7a-bffc-1c21895482ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.400881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa1888-bca1-4c70-87de-69919f8de2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:52.401945 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db82f4-e356-4b7a-bffc-1c21895482ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:52.402372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa1888-bca1-4c70-87de-69919f8de2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.402925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa1888-bca1-4c70-87de-69919f8de2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.403477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa1888-bca1-4c70-87de-69919f8de2a5 admin admin] PATH_INFO: `/v3/projects/891901bf7bd04a3087d1f6aa741eef23/users/4ef639debf0f40189467f98949a4f1ce/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.404696 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db82f4-e356-4b7a-bffc-1c21895482ec admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:52.431394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa1888-bca1-4c70-87de-69919f8de2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891901bf7bd04a3087d1f6aa741eef23, user_id=4ef639debf0f40189467f98949a4f1ce, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:52.435600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa1888-bca1-4c70-87de-69919f8de2a5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:52.447580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 836/1670] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:52.449869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db82f4-e356-4b7a-bffc-1c21895482ec admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:52.488619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 835/1671] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:52 2022] PUT /identity/v3/projects/891901bf7bd04a3087d1f6aa741eef23/users/4ef639debf0f40189467f98949a4f1ce/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:55:52.494092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd6b84-25c0-4111-bf20-4ea60746fce3 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.495948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd6b84-25c0-4111-bf20-4ea60746fce3 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.496273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd6b84-25c0-4111-bf20-4ea60746fce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.505509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db82f4-e356-4b7a-bffc-1c21895482ec admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:52.570701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a8bd6b84-25c0-4111-bf20-4ea60746fce3 None None] MFA Rules not processed for user `4ef639debf0f40189467f98949a4f1ce`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:52.572595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a8bd6b84-25c0-4111-bf20-4ea60746fce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:52.664984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06598ee-a566-4839-baef-d8de0c8e3188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:52.666916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06598ee-a566-4839-baef-d8de0c8e3188 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.666916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06598ee-a566-4839-baef-d8de0c8e3188 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.666916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06598ee-a566-4839-baef-d8de0c8e3188 admin admin] PATH_INFO: `/v3/projects/631ba39d9ea44f5eaefac6a44deac667/users/e5cfa69b2dc24335868a9e4a9e33f2c0/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.729481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06598ee-a566-4839-baef-d8de0c8e3188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=631ba39d9ea44f5eaefac6a44deac667, user_id=e5cfa69b2dc24335868a9e4a9e33f2c0, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:52.734435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06598ee-a566-4839-baef-d8de0c8e3188 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:52.778290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 836/1672] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:52.782231 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4e9e8-11f9-4571-8621-8844e3e7f7f5 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.783429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4e9e8-11f9-4571-8621-8844e3e7f7f5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.783780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4e9e8-11f9-4571-8621-8844e3e7f7f5 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.785772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 837/1673] 10.222.0.79 () {54 vars in 912 bytes} [Fri May 13 13:55:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:55:52.786465 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc3694-89aa-4329-abb7-178740cfd59c None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:52.786799 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc3694-89aa-4329-abb7-178740cfd59c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:52.787007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc3694-89aa-4329-abb7-178740cfd59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:52.802751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 837/1674] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:52 2022] PUT /identity/v3/projects/631ba39d9ea44f5eaefac6a44deac667/users/e5cfa69b2dc24335868a9e4a9e33f2c0/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:55:52.802751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06598ee-a566-4839-baef-d8de0c8e3188 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:52.817145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e06598ee-a566-4839-baef-d8de0c8e3188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:52.861883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-d8fc3694-89aa-4329-abb7-178740cfd59c None None] MFA Rules not processed for user `fff87b1f656a4d3d8559d013711b0e11`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:52.869169 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d8fc3694-89aa-4329-abb7-178740cfd59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:53.061633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 838/1675] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:53.061633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f27a9-bd18-483c-bff7-9ec56b462794 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:53.061633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f27a9-bd18-483c-bff7-9ec56b462794 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:53.061633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f27a9-bd18-483c-bff7-9ec56b462794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:53.071037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06598ee-a566-4839-baef-d8de0c8e3188 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:53.089385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e504b-104f-4a84-bb86-b93d3f1a0b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:53.091219 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e504b-104f-4a84-bb86-b93d3f1a0b33 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:53.091219 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e504b-104f-4a84-bb86-b93d3f1a0b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:53.091491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e504b-104f-4a84-bb86-b93d3f1a0b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:53.151268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7c8f27a9-bd18-483c-bff7-9ec56b462794 None None] MFA Rules not processed for user `8f784cc461ab48ad92140680758a3704`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:53.152561 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7c8f27a9-bd18-483c-bff7-9ec56b462794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:53.260285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e504b-104f-4a84-bb86-b93d3f1a0b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:53.262891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e504b-104f-4a84-bb86-b93d3f1a0b33 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:53.285741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 838/1676] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:53.290098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 839/1677] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:53.298131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86538ce-5e77-4515-8821-82684f5cbdf1 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:53.309739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86538ce-5e77-4515-8821-82684f5cbdf1 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:53.310518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86538ce-5e77-4515-8821-82684f5cbdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:53.336769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f27a9-bd18-483c-bff7-9ec56b462794 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:53.397265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e86538ce-5e77-4515-8821-82684f5cbdf1 None None] MFA Rules not processed for user `e5cfa69b2dc24335868a9e4a9e33f2c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:53.400633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e86538ce-5e77-4515-8821-82684f5cbdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:53.505654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f27a9-bd18-483c-bff7-9ec56b462794 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:53.522582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:53.524122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:53.524621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:53.529695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] PATH_INFO: `/v3/users/4ef639debf0f40189467f98949a4f1ce` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:53.534449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef639debf0f40189467f98949a4f1ce)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:53.544177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:53.584915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 839/1678] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:53.589392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86538ce-5e77-4515-8821-82684f5cbdf1 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:53.617635 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86538ce-5e77-4515-8821-82684f5cbdf1 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:53.632899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338533b-e5e8-464a-8438-b93d3fd58328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:53.633831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338533b-e5e8-464a-8438-b93d3fd58328 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:53.634074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338533b-e5e8-464a-8438-b93d3fd58328 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:53.634352 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338533b-e5e8-464a-8438-b93d3fd58328 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:53.635228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338533b-e5e8-464a-8438-b93d3fd58328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:53.637218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338533b-e5e8-464a-8438-b93d3fd58328 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:53.653194 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef639debf0f40189467f98949a4f1ce'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:53.653938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef639debf0f40189467f98949a4f1ce'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:53.671520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef639debf0f40189467f98949a4f1ce'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:53.672338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef639debf0f40189467f98949a4f1ce'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:53.685619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6e41143f-6c8f-4644-b9f9-85b8780fa838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef639debf0f40189467f98949a4f1ce'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:53.694968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 840/1679] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:53 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:53.699117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338533b-e5e8-464a-8438-b93d3fd58328 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:53.701680 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a338533b-e5e8-464a-8438-b93d3fd58328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:53.725043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 840/1680] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:55:53 2022] DELETE /identity/v3/users/4ef639debf0f40189467f98949a4f1ce => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:55:53.731984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff41f1-f8c6-43ab-84a9-243069c4909c None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:53.732460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff41f1-f8c6-43ab-84a9-243069c4909c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:53.734004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff41f1-f8c6-43ab-84a9-243069c4909c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:53.795813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-85ff41f1-f8c6-43ab-84a9-243069c4909c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:53.801632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-85ff41f1-f8c6-43ab-84a9-243069c4909c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:53.956480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338533b-e5e8-464a-8438-b93d3fd58328 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:54.029344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 841/1681] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:54.104516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56187d70-2b08-4a9a-a001-c57d2996ceef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:54.105367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56187d70-2b08-4a9a-a001-c57d2996ceef admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:54.105536 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56187d70-2b08-4a9a-a001-c57d2996ceef admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:54.106357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56187d70-2b08-4a9a-a001-c57d2996ceef admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:54.106616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56187d70-2b08-4a9a-a001-c57d2996ceef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:54.109663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56187d70-2b08-4a9a-a001-c57d2996ceef admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:54.140197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff41f1-f8c6-43ab-84a9-243069c4909c None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:54.144601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-85ff41f1-f8c6-43ab-84a9-243069c4909c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:54.159631 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 841/1682] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:53 2022] POST /identity/v3/users => generated 289 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:54.161484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56187d70-2b08-4a9a-a001-c57d2996ceef admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:54.187565 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56187d70-2b08-4a9a-a001-c57d2996ceef admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:54.200342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f35e61-be77-4a0e-b5c2-6103d70ec389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:54.201076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f35e61-be77-4a0e-b5c2-6103d70ec389 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:54.201426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f35e61-be77-4a0e-b5c2-6103d70ec389 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:54.201734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f35e61-be77-4a0e-b5c2-6103d70ec389 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:54.202493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f35e61-be77-4a0e-b5c2-6103d70ec389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:54.205936 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f35e61-be77-4a0e-b5c2-6103d70ec389 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:54.235294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 842/1683] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:55:54.235294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f35e61-be77-4a0e-b5c2-6103d70ec389 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:54.239753 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-01f35e61-be77-4a0e-b5c2-6103d70ec389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:54.385798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff41f1-f8c6-43ab-84a9-243069c4909c None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:54.402035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eedf3f-9446-4677-8353-18c55206221b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:54.403948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eedf3f-9446-4677-8353-18c55206221b service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:54.415943 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eedf3f-9446-4677-8353-18c55206221b service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:54.416984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eedf3f-9446-4677-8353-18c55206221b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:54.421455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-88eedf3f-9446-4677-8353-18c55206221b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:54.480616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f35e61-be77-4a0e-b5c2-6103d70ec389 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:54.482556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eedf3f-9446-4677-8353-18c55206221b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:54.490952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eedf3f-9446-4677-8353-18c55206221b service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:54.503263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e074ea-bde4-42af-8860-d562178c2afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:54.507685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e074ea-bde4-42af-8860-d562178c2afd service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:54.508249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e074ea-bde4-42af-8860-d562178c2afd service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:54.508741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e074ea-bde4-42af-8860-d562178c2afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:54.522823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 842/1684] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:55:54.523835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eedf3f-9446-4677-8353-18c55206221b service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:54.552841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eedf3f-9446-4677-8353-18c55206221b service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:54.562994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e074ea-bde4-42af-8860-d562178c2afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:54.565030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e074ea-bde4-42af-8860-d562178c2afd service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:54.566076 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b23f5-04bf-4a9e-a52e-30466de0c481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:54.567561 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b23f5-04bf-4a9e-a52e-30466de0c481 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:54.568325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b23f5-04bf-4a9e-a52e-30466de0c481 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:54.569070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b23f5-04bf-4a9e-a52e-30466de0c481 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:54.570334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b23f5-04bf-4a9e-a52e-30466de0c481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:54.572867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b23f5-04bf-4a9e-a52e-30466de0c481 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:54.600446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 843/1685] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:54.604061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 843/1686] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:54.615295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e074ea-bde4-42af-8860-d562178c2afd service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:54.645431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e074ea-bde4-42af-8860-d562178c2afd service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:54.674254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce7a5a-125d-4f36-b0f3-506fb69cb566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:54.677923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7a5a-125d-4f36-b0f3-506fb69cb566 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:54.678459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7a5a-125d-4f36-b0f3-506fb69cb566 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:54.679114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce7a5a-125d-4f36-b0f3-506fb69cb566 admin admin] PATH_INFO: `/v3/projects/8e4d495b764f4be0aa6535a9f3fbdf08/users/f6a5cae3bece4c6e9f48d3b2c5907545/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:54.743862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce7a5a-125d-4f36-b0f3-506fb69cb566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e4d495b764f4be0aa6535a9f3fbdf08, user_id=f6a5cae3bece4c6e9f48d3b2c5907545, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:54.747907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce7a5a-125d-4f36-b0f3-506fb69cb566 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:54.793236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 844/1687] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:54 2022] PUT /identity/v3/projects/8e4d495b764f4be0aa6535a9f3fbdf08/users/f6a5cae3bece4c6e9f48d3b2c5907545/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:54.806508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353180ac-097c-4419-8f4b-5bf1a62894e7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:54.807911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353180ac-097c-4419-8f4b-5bf1a62894e7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:54.808419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353180ac-097c-4419-8f4b-5bf1a62894e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:54.916098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-353180ac-097c-4419-8f4b-5bf1a62894e7 None None] MFA Rules not processed for user `f6a5cae3bece4c6e9f48d3b2c5907545`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:54.917385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-353180ac-097c-4419-8f4b-5bf1a62894e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:55.150260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 844/1688] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:55.161733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce7a5a-125d-4f36-b0f3-506fb69cb566 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:55.318249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce7a5a-125d-4f36-b0f3-506fb69cb566 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:55.327138 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34069ea6-f1ec-46f4-ac88-0d01741b847f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:55.328307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34069ea6-f1ec-46f4-ac88-0d01741b847f admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:55.329674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34069ea6-f1ec-46f4-ac88-0d01741b847f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:55.330197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34069ea6-f1ec-46f4-ac88-0d01741b847f admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:55.334815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34069ea6-f1ec-46f4-ac88-0d01741b847f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:55.338642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34069ea6-f1ec-46f4-ac88-0d01741b847f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:55.401683 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 845/1689] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:55 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:55.409328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353180ac-097c-4419-8f4b-5bf1a62894e7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:55.427572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353180ac-097c-4419-8f4b-5bf1a62894e7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:55.556772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5966cb-1986-491e-bba5-9c83e4cf336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:55.558041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5966cb-1986-491e-bba5-9c83e4cf336a admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:55.559754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5966cb-1986-491e-bba5-9c83e4cf336a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:55.560242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5966cb-1986-491e-bba5-9c83e4cf336a admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:55.561491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5966cb-1986-491e-bba5-9c83e4cf336a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:55.564314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5966cb-1986-491e-bba5-9c83e4cf336a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:55.608219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 845/1690] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:55 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:55.614132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34069ea6-f1ec-46f4-ac88-0d01741b847f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:55.627304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34069ea6-f1ec-46f4-ac88-0d01741b847f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:55.633924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725ed73-fa40-4060-a6e8-d7043f24820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:55.634619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725ed73-fa40-4060-a6e8-d7043f24820f admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:55.634940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725ed73-fa40-4060-a6e8-d7043f24820f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:55.635256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725ed73-fa40-4060-a6e8-d7043f24820f admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:55.636068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725ed73-fa40-4060-a6e8-d7043f24820f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:55.638962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725ed73-fa40-4060-a6e8-d7043f24820f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:55.657566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 846/1691] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:55.661446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5966cb-1986-491e-bba5-9c83e4cf336a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:55.692861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5966cb-1986-491e-bba5-9c83e4cf336a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:55.710805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515dc19-1b38-49a8-815a-7aeeb8ccdec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:55.714177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515dc19-1b38-49a8-815a-7aeeb8ccdec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:55.714621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515dc19-1b38-49a8-815a-7aeeb8ccdec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:55.714968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515dc19-1b38-49a8-815a-7aeeb8ccdec1 admin admin] PATH_INFO: `/v3/projects/a40e41e735744f7592bd07272829295d/users/710739820a0042be80340adb275d0d1c/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:55.765954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515dc19-1b38-49a8-815a-7aeeb8ccdec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a40e41e735744f7592bd07272829295d, user_id=710739820a0042be80340adb275d0d1c, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:55.770220 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515dc19-1b38-49a8-815a-7aeeb8ccdec1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:55.804313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 846/1692] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:55 2022] PUT /identity/v3/projects/a40e41e735744f7592bd07272829295d/users/710739820a0042be80340adb275d0d1c/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:55.806383 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725ed73-fa40-4060-a6e8-d7043f24820f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:55.820977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725ed73-fa40-4060-a6e8-d7043f24820f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:55.997162 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce048edf-2af4-41ed-8123-da4e4d0ad38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:55.998974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce048edf-2af4-41ed-8123-da4e4d0ad38b admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:55.999651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce048edf-2af4-41ed-8123-da4e4d0ad38b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:56.000125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce048edf-2af4-41ed-8123-da4e4d0ad38b admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:56.002453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce048edf-2af4-41ed-8123-da4e4d0ad38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:56.005215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce048edf-2af4-41ed-8123-da4e4d0ad38b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:56.027670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 847/1693] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:55:56.032162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515dc19-1b38-49a8-815a-7aeeb8ccdec1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:56.051354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515dc19-1b38-49a8-815a-7aeeb8ccdec1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:56.065843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161af1-c1c9-46fd-9329-0fa92fce20e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:56.066792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161af1-c1c9-46fd-9329-0fa92fce20e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:56.067086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161af1-c1c9-46fd-9329-0fa92fce20e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:56.067368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161af1-c1c9-46fd-9329-0fa92fce20e6 admin admin] PATH_INFO: `/v3/projects/a40e41e735744f7592bd07272829295d/users/710739820a0042be80340adb275d0d1c/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:56.073381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb161af1-c1c9-46fd-9329-0fa92fce20e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a40e41e735744f7592bd07272829295d, user_id=710739820a0042be80340adb275d0d1c, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:56.080247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb161af1-c1c9-46fd-9329-0fa92fce20e6 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:56.113562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 847/1694] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:56 2022] PUT /identity/v3/projects/a40e41e735744f7592bd07272829295d/users/710739820a0042be80340adb275d0d1c/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:56.118737 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ccae3-a76e-4529-9566-0b71f3763309 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:56.119056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ccae3-a76e-4529-9566-0b71f3763309 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:56.119304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ccae3-a76e-4529-9566-0b71f3763309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:56.163242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161af1-c1c9-46fd-9329-0fa92fce20e6 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:56.166761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-eb161af1-c1c9-46fd-9329-0fa92fce20e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:56.204484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-693ccae3-a76e-4529-9566-0b71f3763309 None None] MFA Rules not processed for user `710739820a0042be80340adb275d0d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:56.206482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-693ccae3-a76e-4529-9566-0b71f3763309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:56.447188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161af1-c1c9-46fd-9329-0fa92fce20e6 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:56.455726 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 848/1695] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:56.464472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:56.468242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:56.468559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:56.468845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] PATH_INFO: `/v3/projects/891901bf7bd04a3087d1f6aa741eef23` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:56.475742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891901bf7bd04a3087d1f6aa741eef23)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:56.487908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:56.490016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ccae3-a76e-4529-9566-0b71f3763309 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:56.491677 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-693ccae3-a76e-4529-9566-0b71f3763309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:56.626902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891901bf7bd04a3087d1f6aa741eef23', 'request_id': 'req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:56.649571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891901bf7bd04a3087d1f6aa741eef23', 'request_id': 'req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:56.671200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ccae3-a76e-4529-9566-0b71f3763309 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:56.677297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891901bf7bd04a3087d1f6aa741eef23', 'request_id': 'req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:56.714996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] The token cache is being invalidate because project 891901bf7bd04a3087d1f6aa741eef23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:55:56.714996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:56.718012 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cffde3ec-43e4-4c09-b2e5-a7c21a9eb5e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:56.726660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 848/1696] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:55:56 2022] DELETE /identity/v3/projects/891901bf7bd04a3087d1f6aa741eef23 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:56.731444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1e31b-3fa6-451c-a572-a2424a288553 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:56.731790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1e31b-3fa6-451c-a572-a2424a288553 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:56.732704 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1e31b-3fa6-451c-a572-a2424a288553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:56.766126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-30b1e31b-3fa6-451c-a572-a2424a288553 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:56.768621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-30b1e31b-3fa6-451c-a572-a2424a288553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:56.844711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d77bd0-b863-4e49-b4bb-926bf7f9eb16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:56.845971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d77bd0-b863-4e49-b4bb-926bf7f9eb16 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:56.846501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d77bd0-b863-4e49-b4bb-926bf7f9eb16 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:56.847057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d77bd0-b863-4e49-b4bb-926bf7f9eb16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:56.867887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a5d77bd0-b863-4e49-b4bb-926bf7f9eb16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:56.976934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 849/1697] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:56.990594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9d9d1-a466-469d-8ba8-482dc1bed057 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:56.992527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9d9d1-a466-469d-8ba8-482dc1bed057 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:56.999215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9d9d1-a466-469d-8ba8-482dc1bed057 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:57.014621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 850/1698] 10.222.0.79 () {54 vars in 960 bytes} [Fri May 13 13:55:56 2022] GET /identity => generated 270 bytes in 36 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:55:57.017733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cb5b4-3ca1-433a-b296-aff8c3d81676 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:57.018208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cb5b4-3ca1-433a-b296-aff8c3d81676 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:57.027236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cb5b4-3ca1-433a-b296-aff8c3d81676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:57.067172 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d77bd0-b863-4e49-b4bb-926bf7f9eb16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:57.074262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d77bd0-b863-4e49-b4bb-926bf7f9eb16 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:57.077679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e99cb5b4-3ca1-433a-b296-aff8c3d81676 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:57.081979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e99cb5b4-3ca1-433a-b296-aff8c3d81676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:57.110269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 851/1699] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:57.113687 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6790ad5-5130-4c5a-8266-d3c83ede4b9d None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:57.113975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6790ad5-5130-4c5a-8266-d3c83ede4b9d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:57.114251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6790ad5-5130-4c5a-8266-d3c83ede4b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:57.122423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 849/1700] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:55:57.124101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d77bd0-b863-4e49-b4bb-926bf7f9eb16 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:57.163661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d77bd0-b863-4e49-b4bb-926bf7f9eb16 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:57.163924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f6790ad5-5130-4c5a-8266-d3c83ede4b9d None None] MFA Rules not processed for user `4ae9a6f1ac304e21b8d3115f2a0baf88`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:57.168069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f6790ad5-5130-4c5a-8266-d3c83ede4b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:57.178048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4981043-3bd7-4de5-971d-b3d2325ceb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:57.179205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4981043-3bd7-4de5-971d-b3d2325ceb9e admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:57.179599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4981043-3bd7-4de5-971d-b3d2325ceb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:57.179977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4981043-3bd7-4de5-971d-b3d2325ceb9e admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:57.181270 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4981043-3bd7-4de5-971d-b3d2325ceb9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:55:57.183179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4981043-3bd7-4de5-971d-b3d2325ceb9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:57.188796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4981043-3bd7-4de5-971d-b3d2325ceb9e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:57.228156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 850/1701] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:55:57.233499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4981043-3bd7-4de5-971d-b3d2325ceb9e admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:57.274028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4981043-3bd7-4de5-971d-b3d2325ceb9e admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:57.295991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eebc2a-7aaf-482f-9501-8d63b553dec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:57.296785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eebc2a-7aaf-482f-9501-8d63b553dec7 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:57.297318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eebc2a-7aaf-482f-9501-8d63b553dec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:57.297682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eebc2a-7aaf-482f-9501-8d63b553dec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:57.299195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eebc2a-7aaf-482f-9501-8d63b553dec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:57.301885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eebc2a-7aaf-482f-9501-8d63b553dec7 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:57.369457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 851/1702] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:55:57 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:57.375319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eebc2a-7aaf-482f-9501-8d63b553dec7 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:57.388299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f9eebc2a-7aaf-482f-9501-8d63b553dec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:57.399650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 852/1703] 10.222.0.79 () {58 vars in 1057 bytes} [Fri May 13 13:55:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:57.401529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6790ad5-5130-4c5a-8266-d3c83ede4b9d None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:57.432797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6790ad5-5130-4c5a-8266-d3c83ede4b9d None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:57.519633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eebc2a-7aaf-482f-9501-8d63b553dec7 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:57.534595 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352a649-e6e6-4a27-a965-fa0a20accd01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:57.536640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352a649-e6e6-4a27-a965-fa0a20accd01 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:57.537065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352a649-e6e6-4a27-a965-fa0a20accd01 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:57.537428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352a649-e6e6-4a27-a965-fa0a20accd01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:57.541147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f352a649-e6e6-4a27-a965-fa0a20accd01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:57.687135 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352a649-e6e6-4a27-a965-fa0a20accd01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:57.689684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352a649-e6e6-4a27-a965-fa0a20accd01 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:57.715614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 852/1704] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:57.718534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352a649-e6e6-4a27-a965-fa0a20accd01 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:57.781215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33638c29-1649-4aa6-b6b7-00f1d190e101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:57.782449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33638c29-1649-4aa6-b6b7-00f1d190e101 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:57.783101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33638c29-1649-4aa6-b6b7-00f1d190e101 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:57.783616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33638c29-1649-4aa6-b6b7-00f1d190e101 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:57.784978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33638c29-1649-4aa6-b6b7-00f1d190e101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:57.805918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33638c29-1649-4aa6-b6b7-00f1d190e101 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:57.848852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352a649-e6e6-4a27-a965-fa0a20accd01 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:57.879978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed48edf-d05f-4c44-906f-5800847c99ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ae9a6f1ac304e21b8d3115f2a0baf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:57.883627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed48edf-d05f-4c44-906f-5800847c99ca service placement] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:57.884266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed48edf-d05f-4c44-906f-5800847c99ca service placement] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:57.901078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed48edf-d05f-4c44-906f-5800847c99ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:57.922024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-aed48edf-d05f-4c44-906f-5800847c99ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:57.944102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 853/1705] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:55:57 2022] POST /identity/v3/users => generated 286 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:55:57.956781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33638c29-1649-4aa6-b6b7-00f1d190e101 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:57.980770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed48edf-d05f-4c44-906f-5800847c99ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:57.982697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed48edf-d05f-4c44-906f-5800847c99ca service placement] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:58.011984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 853/1706] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:55:58.066088 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33638c29-1649-4aa6-b6b7-00f1d190e101 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:58.090423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2f4a0-96cb-4ad6-a3a5-bbe3bddc5bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:58.091171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2f4a0-96cb-4ad6-a3a5-bbe3bddc5bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:58.091500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2f4a0-96cb-4ad6-a3a5-bbe3bddc5bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:58.091870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2f4a0-96cb-4ad6-a3a5-bbe3bddc5bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:58.092630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2f4a0-96cb-4ad6-a3a5-bbe3bddc5bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:58.094341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2f4a0-96cb-4ad6-a3a5-bbe3bddc5bd2 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:58.130069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 854/1707] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:55:58.134361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed48edf-d05f-4c44-906f-5800847c99ca service placement] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:58.159293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed48edf-d05f-4c44-906f-5800847c99ca service placement] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:58.193016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548c817-5a13-4fc4-8205-f205187e9748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:58.193769 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548c817-5a13-4fc4-8205-f205187e9748 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:58.194055 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548c817-5a13-4fc4-8205-f205187e9748 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:58.194335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548c817-5a13-4fc4-8205-f205187e9748 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:58.195118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548c817-5a13-4fc4-8205-f205187e9748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:58.196796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548c817-5a13-4fc4-8205-f205187e9748 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:58.228863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 854/1708] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:55:58.233742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2f4a0-96cb-4ad6-a3a5-bbe3bddc5bd2 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:58.254603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2f4a0-96cb-4ad6-a3a5-bbe3bddc5bd2 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:58.283426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e331b-01db-4999-8c86-fa83c59672d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:58.284443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e331b-01db-4999-8c86-fa83c59672d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:58.284899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e331b-01db-4999-8c86-fa83c59672d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:58.285235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e331b-01db-4999-8c86-fa83c59672d4 admin admin] PATH_INFO: `/v3/projects/c0ff6e5432c640d08f5c9d1bf950493d/users/9b78cf89050945c3998ae347060791be/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:58.342948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e331b-01db-4999-8c86-fa83c59672d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ff6e5432c640d08f5c9d1bf950493d, user_id=9b78cf89050945c3998ae347060791be, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:58.349712 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e331b-01db-4999-8c86-fa83c59672d4 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:58.381716 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 855/1709] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:55:58 2022] PUT /identity/v3/projects/c0ff6e5432c640d08f5c9d1bf950493d/users/9b78cf89050945c3998ae347060791be/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:55:58.394499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc2372-ffab-450f-9f0a-a84210cb5592 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:58.394860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc2372-ffab-450f-9f0a-a84210cb5592 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:58.395130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc2372-ffab-450f-9f0a-a84210cb5592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:58.468831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f9dc2372-ffab-450f-9f0a-a84210cb5592 None None] MFA Rules not processed for user `9b78cf89050945c3998ae347060791be`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:58.470658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f9dc2372-ffab-450f-9f0a-a84210cb5592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:58.680757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 855/1710] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:58.693293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e331b-01db-4999-8c86-fa83c59672d4 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:58.695702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6d6e331b-01db-4999-8c86-fa83c59672d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:58.861538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e331b-01db-4999-8c86-fa83c59672d4 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:58.881104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f3de4-5ec1-4729-8308-3a1f19b492ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:58.881940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f3de4-5ec1-4729-8308-3a1f19b492ff service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:58.883059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f3de4-5ec1-4729-8308-3a1f19b492ff service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:58.883377 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f3de4-5ec1-4729-8308-3a1f19b492ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:58.905166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f3de4-5ec1-4729-8308-3a1f19b492ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:58.905789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f3de4-5ec1-4729-8308-3a1f19b492ff service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:58.941876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 856/1711] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:55:58.957483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc2372-ffab-450f-9f0a-a84210cb5592 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:59.114931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc2372-ffab-450f-9f0a-a84210cb5592 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:55:59.123735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:55:59.125065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:59.125501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:59.126019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] PATH_INFO: `/v3/users/9b78cf89050945c3998ae347060791be` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:59.128790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b78cf89050945c3998ae347060791be)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:55:59.131563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:55:59.258094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b78cf89050945c3998ae347060791be'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:59.315044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b78cf89050945c3998ae347060791be'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:59.351525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b78cf89050945c3998ae347060791be'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:59.354899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b78cf89050945c3998ae347060791be'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:59.386209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b78cf89050945c3998ae347060791be'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:55:59.432754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 856/1712] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:55:58 2022] DELETE /identity/v3/users/9b78cf89050945c3998ae347060791be => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:55:59.439910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147e38f-50cf-4a1e-825c-aa11a2f62308 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:55:59.440344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147e38f-50cf-4a1e-825c-aa11a2f62308 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:55:59.440786 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147e38f-50cf-4a1e-825c-aa11a2f62308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:55:59.480141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-8147e38f-50cf-4a1e-825c-aa11a2f62308 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:55:59.486900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8147e38f-50cf-4a1e-825c-aa11a2f62308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:55:59.695459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 857/1713] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:55:59.850454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:55:59.852134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:00.069000 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792a710-0c1c-42a5-b0d6-a9fcf20b90c0 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:00.090417 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9161a2-4ce3-4c23-bd2b-cf6430467f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:00.092107 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9161a2-4ce3-4c23-bd2b-cf6430467f44 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:00.092835 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9161a2-4ce3-4c23-bd2b-cf6430467f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:00.093657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9161a2-4ce3-4c23-bd2b-cf6430467f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:00.140317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9161a2-4ce3-4c23-bd2b-cf6430467f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:00.143950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9161a2-4ce3-4c23-bd2b-cf6430467f44 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:00.183112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 857/1714] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:01.820847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147e38f-50cf-4a1e-825c-aa11a2f62308 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:01.828515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8147e38f-50cf-4a1e-825c-aa11a2f62308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:01.854800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147e38f-50cf-4a1e-825c-aa11a2f62308 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:01.864004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:01.865234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:01.865699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:01.866171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] PATH_INFO: `/v3/users/c672faa2a3d849adb0898cb11b5e5038` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:01.867715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c672faa2a3d849adb0898cb11b5e5038)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:01.871180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:01.995898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a75bd-970b-4054-bd63-0a24191e7c8d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:01.995898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a75bd-970b-4054-bd63-0a24191e7c8d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:01.995898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a75bd-970b-4054-bd63-0a24191e7c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:01.997005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c672faa2a3d849adb0898cb11b5e5038'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:01.997005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c672faa2a3d849adb0898cb11b5e5038'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:02.022788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c672faa2a3d849adb0898cb11b5e5038'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:02.023882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c672faa2a3d849adb0898cb11b5e5038'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:02.034671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c672faa2a3d849adb0898cb11b5e5038'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:02.060150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e82a75bd-970b-4054-bd63-0a24191e7c8d None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:02.062595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 858/1715] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:56:01 2022] DELETE /identity/v3/users/c672faa2a3d849adb0898cb11b5e5038 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:02.067776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e82a75bd-970b-4054-bd63-0a24191e7c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:02.265993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 858/1716] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 13:56:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:02.549538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:02.552325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:02.759343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031e9ec-02e5-46d3-b3ff-86b8f8fc9a45 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:02.776592 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a495b7-c327-4360-b33b-42ccefa71902 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:02.777319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a495b7-c327-4360-b33b-42ccefa71902 service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:02.777646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a495b7-c327-4360-b33b-42ccefa71902 service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:02.777930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a495b7-c327-4360-b33b-42ccefa71902 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:02.801578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a495b7-c327-4360-b33b-42ccefa71902 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:02.804602 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a495b7-c327-4360-b33b-42ccefa71902 service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:02.830088 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 859/1717] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 13:56:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:56:02.994340 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a75bd-970b-4054-bd63-0a24191e7c8d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:02.996555 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e82a75bd-970b-4054-bd63-0a24191e7c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:03.006132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a495b7-c327-4360-b33b-42ccefa71902 service swift] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:03.007669 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a4a495b7-c327-4360-b33b-42ccefa71902 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:03.218930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a75bd-970b-4054-bd63-0a24191e7c8d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:03.229987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:03.237521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:03.238200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:03.238427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] PATH_INFO: `/v3/projects/c0ff6e5432c640d08f5c9d1bf950493d` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:03.240333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ff6e5432c640d08f5c9d1bf950493d)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:03.242413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:03.362578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ff6e5432c640d08f5c9d1bf950493d', 'request_id': 'req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:03.373842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ff6e5432c640d08f5c9d1bf950493d', 'request_id': 'req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:03.380041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a495b7-c327-4360-b33b-42ccefa71902 service swift] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:03.389442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ff6e5432c640d08f5c9d1bf950493d', 'request_id': 'req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:03.409640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] The token cache is being invalidate because project c0ff6e5432c640d08f5c9d1bf950493d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:03.410601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:03.412055 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9d02c84b-5f6d-4f1e-b0be-07719bf66cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:03.414230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 859/1718] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:02 2022] DELETE /identity/v3/projects/c0ff6e5432c640d08f5c9d1bf950493d => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:03.421544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f358598-ba8d-4f7b-9f0d-a824a05c8d31 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:03.422410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f358598-ba8d-4f7b-9f0d-a824a05c8d31 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:03.428980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f358598-ba8d-4f7b-9f0d-a824a05c8d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:03.481498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2f358598-ba8d-4f7b-9f0d-a824a05c8d31 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:03.485412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2f358598-ba8d-4f7b-9f0d-a824a05c8d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:03.654812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 860/1719] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:03.672192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc6ffe-7605-436a-a4d2-9978d617b175 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:03.676019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc6ffe-7605-436a-a4d2-9978d617b175 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:03.677198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc6ffe-7605-436a-a4d2-9978d617b175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:03.709071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:03.709417 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-62dc6ffe-7605-436a-a4d2-9978d617b175 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:03.711895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-62dc6ffe-7605-436a-a4d2-9978d617b175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:03.712044 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:03.712659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:03.713223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] PATH_INFO: `/v3/projects/b95ab7620c68480195a6e9505fdacfec` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:03.718597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=b95ab7620c68480195a6e9505fdacfec)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:03.722864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:03.723854 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 861/1720] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:03.725658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fba58a-3704-42a1-966b-8aeffa7da16a None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:03.726123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fba58a-3704-42a1-966b-8aeffa7da16a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:03.726981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fba58a-3704-42a1-966b-8aeffa7da16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:03.763461 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f9fba58a-3704-42a1-966b-8aeffa7da16a None None] MFA Rules not processed for user `648ae92126944b50a22ad780e92df914`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:03.765061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f9fba58a-3704-42a1-966b-8aeffa7da16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:03.912172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b95ab7620c68480195a6e9505fdacfec', 'request_id': 'req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:03.972509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b95ab7620c68480195a6e9505fdacfec', 'request_id': 'req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:04.008876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 862/1721] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:04.009799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fba58a-3704-42a1-966b-8aeffa7da16a None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:04.041260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b95ab7620c68480195a6e9505fdacfec', 'request_id': 'req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:04.080187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fba58a-3704-42a1-966b-8aeffa7da16a None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:04.082335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] The token cache is being invalidate because project b95ab7620c68480195a6e9505fdacfec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:04.082636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:04.096001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:04.101242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 860/1722] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:03 2022] DELETE /identity/v3/projects/b95ab7620c68480195a6e9505fdacfec => generated 0 bytes in 1096 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:56:04.107005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:04.115790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:04.316706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c3fc2-aa6b-477d-911b-58e00c1c0bff admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:04.328577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fd420-020c-4329-9993-195b39ee6886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:04.330120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fd420-020c-4329-9993-195b39ee6886 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:04.330626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fd420-020c-4329-9993-195b39ee6886 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:04.331032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fd420-020c-4329-9993-195b39ee6886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:04.345096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e95fd420-020c-4329-9993-195b39ee6886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:04.425205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691d1e3-ef72-40e2-a81a-3c82e7c57f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:04.425919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691d1e3-ef72-40e2-a81a-3c82e7c57f84 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:04.426195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691d1e3-ef72-40e2-a81a-3c82e7c57f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:04.426518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691d1e3-ef72-40e2-a81a-3c82e7c57f84 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:04.427339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691d1e3-ef72-40e2-a81a-3c82e7c57f84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:56:04.427656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691d1e3-ef72-40e2-a81a-3c82e7c57f84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:04.430200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691d1e3-ef72-40e2-a81a-3c82e7c57f84 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:04.501655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 863/1723] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 492 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:56:04.503083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691d1e3-ef72-40e2-a81a-3c82e7c57f84 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:04.522032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0691d1e3-ef72-40e2-a81a-3c82e7c57f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:04.577367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691d1e3-ef72-40e2-a81a-3c82e7c57f84 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:04.613329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fd420-020c-4329-9993-195b39ee6886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:04.621948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fd420-020c-4329-9993-195b39ee6886 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:04.637848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:04.640179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:04.642105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:04.647202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] PATH_INFO: `/v3/users/fff87b1f656a4d3d8559d013711b0e11` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:04.650813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] RBAC: Authorizing `identity:delete_user(user_id=fff87b1f656a4d3d8559d013711b0e11)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:04.653542 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:04.690411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 861/1724] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:04.692203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fd420-020c-4329-9993-195b39ee6886 service nova] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:04.698485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e95fd420-020c-4329-9993-195b39ee6886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:04.779309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fd420-020c-4329-9993-195b39ee6886 service nova] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:04.800726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516e814-4406-4961-96fe-b36ce6dec7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:04.801674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516e814-4406-4961-96fe-b36ce6dec7bb admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:04.802060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516e814-4406-4961-96fe-b36ce6dec7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:04.802467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516e814-4406-4961-96fe-b36ce6dec7bb admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:04.803371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516e814-4406-4961-96fe-b36ce6dec7bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:04.805607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516e814-4406-4961-96fe-b36ce6dec7bb admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:04.882230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 862/1725] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:56:04 2022] POST /identity/v3/projects => generated 361 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:04.885018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516e814-4406-4961-96fe-b36ce6dec7bb admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:04.888564 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6516e814-4406-4961-96fe-b36ce6dec7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:04.893456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fff87b1f656a4d3d8559d013711b0e11'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:04.928754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fff87b1f656a4d3d8559d013711b0e11'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:04.952136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fff87b1f656a4d3d8559d013711b0e11'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:04.953972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fff87b1f656a4d3d8559d013711b0e11'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:04.994526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fff87b1f656a4d3d8559d013711b0e11'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:05.011441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 864/1726] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:56:04 2022] DELETE /identity/v3/users/fff87b1f656a4d3d8559d013711b0e11 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:05.013170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:05.016529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:05.036248 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516e814-4406-4961-96fe-b36ce6dec7bb admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:05.252466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:05.254387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:05.255557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:05.256271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] PATH_INFO: `/v3/users/e5cfa69b2dc24335868a9e4a9e33f2c0` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:05.258829 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5cfa69b2dc24335868a9e4a9e33f2c0)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:05.262574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:05.323361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d102a-e4ed-4ad5-b8ca-008d8d27f3db admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:05.352562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3f564-c6a2-4fcb-a7e3-8cdab755083b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:05.353283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3f564-c6a2-4fcb-a7e3-8cdab755083b admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:05.353568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3f564-c6a2-4fcb-a7e3-8cdab755083b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:05.353851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3f564-c6a2-4fcb-a7e3-8cdab755083b admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:05.354569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3f564-c6a2-4fcb-a7e3-8cdab755083b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:05.357152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3f564-c6a2-4fcb-a7e3-8cdab755083b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:05.432324 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5cfa69b2dc24335868a9e4a9e33f2c0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:05.436637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5cfa69b2dc24335868a9e4a9e33f2c0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:05.455072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 865/1727] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:56:05 2022] POST /identity/v3/users => generated 281 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:05.463039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3f564-c6a2-4fcb-a7e3-8cdab755083b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:05.491855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5cfa69b2dc24335868a9e4a9e33f2c0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:05.493980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3f564-c6a2-4fcb-a7e3-8cdab755083b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:05.494611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5cfa69b2dc24335868a9e4a9e33f2c0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:05.513437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f21c1-aaad-4eda-991f-82df383d5d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:05.515357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f21c1-aaad-4eda-991f-82df383d5d7b admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:05.517424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f21c1-aaad-4eda-991f-82df383d5d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:05.518043 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f21c1-aaad-4eda-991f-82df383d5d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:05.521862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f21c1-aaad-4eda-991f-82df383d5d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:05.528790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-df998072-7ea8-417b-91af-e33925b7e24f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5cfa69b2dc24335868a9e4a9e33f2c0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:05.530975 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f21c1-aaad-4eda-991f-82df383d5d7b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:05.571154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 866/1728] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:05.572305 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f21c1-aaad-4eda-991f-82df383d5d7b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:05.577924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 863/1729] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:56:04 2022] DELETE /identity/v3/users/e5cfa69b2dc24335868a9e4a9e33f2c0 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:05.580610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1b1f21c1-aaad-4eda-991f-82df383d5d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:05.584666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800f82b-378e-44c0-9173-b5eb3d1da8d4 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:05.585118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800f82b-378e-44c0-9173-b5eb3d1da8d4 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:05.585648 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800f82b-378e-44c0-9173-b5eb3d1da8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:05.625022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f21c1-aaad-4eda-991f-82df383d5d7b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:05.656202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a800f82b-378e-44c0-9173-b5eb3d1da8d4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:05.660314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a800f82b-378e-44c0-9173-b5eb3d1da8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:05.863710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 864/1730] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:05.872470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68899ba7-97f2-4ec8-a21a-0a054f6c63ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:05.873627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68899ba7-97f2-4ec8-a21a-0a054f6c63ca admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:05.874007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68899ba7-97f2-4ec8-a21a-0a054f6c63ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:05.874387 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68899ba7-97f2-4ec8-a21a-0a054f6c63ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:05.875388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68899ba7-97f2-4ec8-a21a-0a054f6c63ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:05.878054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68899ba7-97f2-4ec8-a21a-0a054f6c63ca admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:05.920442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 867/1731] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:05.924913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800f82b-378e-44c0-9173-b5eb3d1da8d4 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:05.996263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800f82b-378e-44c0-9173-b5eb3d1da8d4 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:06.020823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ef4bd-a7b7-42c3-a8d3-49edc424d521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:06.023787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ef4bd-a7b7-42c3-a8d3-49edc424d521 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:06.024549 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ef4bd-a7b7-42c3-a8d3-49edc424d521 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:06.025347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ef4bd-a7b7-42c3-a8d3-49edc424d521 admin admin] PATH_INFO: `/v3/projects/cc9d3d16e75342b1a000f597fead9f04/users/3a81c0fa4d9c433898221f22a8626ba3/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:06.064598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ef4bd-a7b7-42c3-a8d3-49edc424d521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc9d3d16e75342b1a000f597fead9f04, user_id=3a81c0fa4d9c433898221f22a8626ba3, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:06.090069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ef4bd-a7b7-42c3-a8d3-49edc424d521 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:06.100521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68899ba7-97f2-4ec8-a21a-0a054f6c63ca admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:06.102699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-68899ba7-97f2-4ec8-a21a-0a054f6c63ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:06.178938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 865/1732] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:05 2022] PUT /identity/v3/projects/cc9d3d16e75342b1a000f597fead9f04/users/3a81c0fa4d9c433898221f22a8626ba3/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:06.185815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2d6d8-691c-42e7-b4f7-249033faeb9c None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:06.186158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2d6d8-691c-42e7-b4f7-249033faeb9c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:06.186514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2d6d8-691c-42e7-b4f7-249033faeb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:06.305630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-81c2d6d8-691c-42e7-b4f7-249033faeb9c None None] MFA Rules not processed for user `3a81c0fa4d9c433898221f22a8626ba3`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:06.307035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-81c2d6d8-691c-42e7-b4f7-249033faeb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:06.418247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68899ba7-97f2-4ec8-a21a-0a054f6c63ca admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:06.711943 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 866/1733] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 530 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:06.716813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d6c3b-66c2-43fc-a3d0-cbe953eac18d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:06.721139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2d6d8-691c-42e7-b4f7-249033faeb9c None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:06.722302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6c3b-66c2-43fc-a3d0-cbe953eac18d service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:06.728805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6c3b-66c2-43fc-a3d0-cbe953eac18d service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:06.729332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6c3b-66c2-43fc-a3d0-cbe953eac18d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:06.733559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-81c2d6d8-691c-42e7-b4f7-249033faeb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:06.961975 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d6c3b-66c2-43fc-a3d0-cbe953eac18d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:06.963885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d6c3b-66c2-43fc-a3d0-cbe953eac18d service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:07.002866 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 868/1734] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 903 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:07.004978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d6c3b-66c2-43fc-a3d0-cbe953eac18d service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:07.008507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c46d6c3b-66c2-43fc-a3d0-cbe953eac18d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:07.034051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2d6d8-691c-42e7-b4f7-249033faeb9c None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:07.041985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:07.051103 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:07.051667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:07.052109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] PATH_INFO: `/v3/projects/43e71917c48246fab1030a732f2616ee` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:07.057492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=43e71917c48246fab1030a732f2616ee)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:07.060292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:07.197441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d6c3b-66c2-43fc-a3d0-cbe953eac18d service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:07.233827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43e71917c48246fab1030a732f2616ee', 'request_id': 'req-17c486f6-0097-4c07-af6d-36edc396fb1e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:07.267338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43e71917c48246fab1030a732f2616ee', 'request_id': 'req-17c486f6-0097-4c07-af6d-36edc396fb1e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:07.294991 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43e71917c48246fab1030a732f2616ee', 'request_id': 'req-17c486f6-0097-4c07-af6d-36edc396fb1e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:07.326455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb10163-40f5-4c8a-beb1-dd7e2674db92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:07.327500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb10163-40f5-4c8a-beb1-dd7e2674db92 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:07.328169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb10163-40f5-4c8a-beb1-dd7e2674db92 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:07.345114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb10163-40f5-4c8a-beb1-dd7e2674db92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:07.365847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] The token cache is being invalidate because project 43e71917c48246fab1030a732f2616ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:07.366294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:07.367163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-17c486f6-0097-4c07-af6d-36edc396fb1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:07.368822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 867/1735] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:06 2022] DELETE /identity/v3/projects/43e71917c48246fab1030a732f2616ee => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:07.378812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717054d-a07c-4dca-ac78-8fedfaaabc14 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:07.379321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717054d-a07c-4dca-ac78-8fedfaaabc14 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:07.379794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717054d-a07c-4dca-ac78-8fedfaaabc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:07.444504 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-6717054d-a07c-4dca-ac78-8fedfaaabc14 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:07.446190 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6717054d-a07c-4dca-ac78-8fedfaaabc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:07.715571 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb10163-40f5-4c8a-beb1-dd7e2674db92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:07.726836 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb10163-40f5-4c8a-beb1-dd7e2674db92 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:07.732587 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 868/1736] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:56:07.734208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afd5a5-29ee-4f89-b627-34226554fcf7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:07.734507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afd5a5-29ee-4f89-b627-34226554fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:07.734835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afd5a5-29ee-4f89-b627-34226554fcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:07.783721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 869/1737] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 780 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:07.801736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb10163-40f5-4c8a-beb1-dd7e2674db92 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:07.854391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb10163-40f5-4c8a-beb1-dd7e2674db92 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:07.873010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-00afd5a5-29ee-4f89-b627-34226554fcf7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:07.873371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:07.874413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-00afd5a5-29ee-4f89-b627-34226554fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:07.874700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:07.875161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:07.875580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] PATH_INFO: `/v3/users/3a81c0fa4d9c433898221f22a8626ba3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:07.883469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a81c0fa4d9c433898221f22a8626ba3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:07.885705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:07.931349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 869/1738] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:07.943910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afd5a5-29ee-4f89-b627-34226554fcf7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:07.997209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afd5a5-29ee-4f89-b627-34226554fcf7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:08.035886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31693-0443-4589-bcb1-c2ea802f5898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:08.036299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31693-0443-4589-bcb1-c2ea802f5898 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:08.036586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31693-0443-4589-bcb1-c2ea802f5898 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:08.036863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c31693-0443-4589-bcb1-c2ea802f5898 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:08.037623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c31693-0443-4589-bcb1-c2ea802f5898 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:56:08.037945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c31693-0443-4589-bcb1-c2ea802f5898 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:08.057450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c31693-0443-4589-bcb1-c2ea802f5898 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:08.117182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 870/1739] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:56:08.126265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31693-0443-4589-bcb1-c2ea802f5898 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:08.169720 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c31693-0443-4589-bcb1-c2ea802f5898 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:08.191362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055e5fd-4201-48f3-ac81-fd99b1c83a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:08.195752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055e5fd-4201-48f3-ac81-fd99b1c83a3e admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:08.196394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055e5fd-4201-48f3-ac81-fd99b1c83a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:08.201146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055e5fd-4201-48f3-ac81-fd99b1c83a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:08.202265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055e5fd-4201-48f3-ac81-fd99b1c83a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:08.206416 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055e5fd-4201-48f3-ac81-fd99b1c83a3e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:08.284086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 871/1740] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:56:08 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:08.290092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055e5fd-4201-48f3-ac81-fd99b1c83a3e admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:08.417327 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055e5fd-4201-48f3-ac81-fd99b1c83a3e admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:08.441865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77840ac5-cfd6-4306-a72a-7262cb3117a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:08.444816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77840ac5-cfd6-4306-a72a-7262cb3117a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:08.445200 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77840ac5-cfd6-4306-a72a-7262cb3117a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:08.446603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77840ac5-cfd6-4306-a72a-7262cb3117a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:08.455465 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77840ac5-cfd6-4306-a72a-7262cb3117a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:08.457595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77840ac5-cfd6-4306-a72a-7262cb3117a7 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:08.475717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a81c0fa4d9c433898221f22a8626ba3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:08.515204 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 872/1741] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:56:08 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:08.520232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77840ac5-cfd6-4306-a72a-7262cb3117a7 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:08.523923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a81c0fa4d9c433898221f22a8626ba3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:08.538547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a81c0fa4d9c433898221f22a8626ba3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:08.541600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a81c0fa4d9c433898221f22a8626ba3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:08.553845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77840ac5-cfd6-4306-a72a-7262cb3117a7 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:08.562395 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bb3bc82b-6023-4f34-81a2-34efc7bd5e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a81c0fa4d9c433898221f22a8626ba3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:08.567528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7356fa-1c84-4c3f-a26c-a695d836d1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:08.567528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7356fa-1c84-4c3f-a26c-a695d836d1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:08.567951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7356fa-1c84-4c3f-a26c-a695d836d1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:08.567951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7356fa-1c84-4c3f-a26c-a695d836d1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:08.569044 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7356fa-1c84-4c3f-a26c-a695d836d1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:08.578038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7356fa-1c84-4c3f-a26c-a695d836d1c4 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:08.578905 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 870/1742] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:56:07 2022] DELETE /identity/v3/users/3a81c0fa4d9c433898221f22a8626ba3 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:08.584808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb050e82-815d-4b69-89c7-abbb2ffb4020 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:08.585017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb050e82-815d-4b69-89c7-abbb2ffb4020 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:08.585614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb050e82-815d-4b69-89c7-abbb2ffb4020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:08.599139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 873/1743] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:08.600351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7356fa-1c84-4c3f-a26c-a695d836d1c4 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:08.603729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4c7356fa-1c84-4c3f-a26c-a695d836d1c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:08.652811 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-bb050e82-815d-4b69-89c7-abbb2ffb4020 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:08.654068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-bb050e82-815d-4b69-89c7-abbb2ffb4020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:08.663754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7356fa-1c84-4c3f-a26c-a695d836d1c4 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:08.804797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fbfe0-b835-4df9-aab3-8efc0bee564f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:08.805438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fbfe0-b835-4df9-aab3-8efc0bee564f admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:08.805723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fbfe0-b835-4df9-aab3-8efc0bee564f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:08.806000 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fbfe0-b835-4df9-aab3-8efc0bee564f admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:08.806760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fbfe0-b835-4df9-aab3-8efc0bee564f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:08.813067 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fbfe0-b835-4df9-aab3-8efc0bee564f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:08.816683 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 871/1744] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:08.845071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb050e82-815d-4b69-89c7-abbb2ffb4020 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:08.853579 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 874/1745] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:08.858677 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb050e82-815d-4b69-89c7-abbb2ffb4020 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:08.870871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5b48e-f38a-45eb-883f-14e79c96b746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:08.872587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5b48e-f38a-45eb-883f-14e79c96b746 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:08.872873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5b48e-f38a-45eb-883f-14e79c96b746 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:08.873147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5b48e-f38a-45eb-883f-14e79c96b746 admin admin] PATH_INFO: `/v3/projects/31a154cba8db4ed58594e81e7028648e/users/b0cbbded7575465f9576def2948e12de/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:08.948135 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5b48e-f38a-45eb-883f-14e79c96b746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a154cba8db4ed58594e81e7028648e, user_id=b0cbbded7575465f9576def2948e12de, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:08.956212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5b48e-f38a-45eb-883f-14e79c96b746 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:09.002862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 872/1746] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:08 2022] PUT /identity/v3/projects/31a154cba8db4ed58594e81e7028648e/users/b0cbbded7575465f9576def2948e12de/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:56:09.014284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff0a68-5dba-4ca5-8181-8a097e9fbfa5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:09.016246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff0a68-5dba-4ca5-8181-8a097e9fbfa5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:09.016981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff0a68-5dba-4ca5-8181-8a097e9fbfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:09.066994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-88ff0a68-5dba-4ca5-8181-8a097e9fbfa5 None None] MFA Rules not processed for user `b0cbbded7575465f9576def2948e12de`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:09.074153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-88ff0a68-5dba-4ca5-8181-8a097e9fbfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:09.099822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5b48e-f38a-45eb-883f-14e79c96b746 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:09.101937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0fa5b48e-f38a-45eb-883f-14e79c96b746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:09.289581 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5b48e-f38a-45eb-883f-14e79c96b746 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:09.313611 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204d2f8-7c50-43b5-a884-490e1644a043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:09.317390 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204d2f8-7c50-43b5-a884-490e1644a043 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:09.319288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204d2f8-7c50-43b5-a884-490e1644a043 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:09.319736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204d2f8-7c50-43b5-a884-490e1644a043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:09.396975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 875/1747] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:09.408697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff0a68-5dba-4ca5-8181-8a097e9fbfa5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:09.466635 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204d2f8-7c50-43b5-a884-490e1644a043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:09.468760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204d2f8-7c50-43b5-a884-490e1644a043 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:09.510944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 873/1748] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:56:09.616310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff0a68-5dba-4ca5-8181-8a097e9fbfa5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:09.632869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23c62a-9d8f-4d8f-ab42-0a6c5f9c36e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:09.635501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23c62a-9d8f-4d8f-ab42-0a6c5f9c36e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:09.636070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23c62a-9d8f-4d8f-ab42-0a6c5f9c36e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:09.636597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23c62a-9d8f-4d8f-ab42-0a6c5f9c36e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:09.637580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23c62a-9d8f-4d8f-ab42-0a6c5f9c36e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:09.644130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23c62a-9d8f-4d8f-ab42-0a6c5f9c36e3 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:09.669694 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a4b40-cca9-4afc-85eb-e9fb2fc6e576 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:09.669987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a4b40-cca9-4afc-85eb-e9fb2fc6e576 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:09.670425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a4b40-cca9-4afc-85eb-e9fb2fc6e576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:09.717149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 876/1749] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:56:09 2022] POST /identity/v3/projects => generated 371 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:09.722387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23c62a-9d8f-4d8f-ab42-0a6c5f9c36e3 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:09.724058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-fa5a4b40-cca9-4afc-85eb-e9fb2fc6e576 None None] MFA Rules not processed for user `8f784cc461ab48ad92140680758a3704`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:09.730166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-fa5a4b40-cca9-4afc-85eb-e9fb2fc6e576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:09.931731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 874/1750] 10.222.0.87 () {58 vars in 1015 bytes} [Fri May 13 13:56:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:56:09.944055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23c62a-9d8f-4d8f-ab42-0a6c5f9c36e3 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:09.956797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e0c8f-309d-4f7f-9212-8bc493d05e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:09.959064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e0c8f-309d-4f7f-9212-8bc493d05e72 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:09.959708 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e0c8f-309d-4f7f-9212-8bc493d05e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:09.960186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e0c8f-309d-4f7f-9212-8bc493d05e72 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:09.961605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e0c8f-309d-4f7f-9212-8bc493d05e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:09.965212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e0c8f-309d-4f7f-9212-8bc493d05e72 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:10.005297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 877/1751] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:56:09 2022] POST /identity/v3/users => generated 292 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:10.007582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a4b40-cca9-4afc-85eb-e9fb2fc6e576 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:10.030418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a4b40-cca9-4afc-85eb-e9fb2fc6e576 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:10.042135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8bc3b-fc2d-42f7-b1be-7fff9b9019a0 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:10.042959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8bc3b-fc2d-42f7-b1be-7fff9b9019a0 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:10.043535 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8bc3b-fc2d-42f7-b1be-7fff9b9019a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:10.046024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dab1f6-0782-4122-9cbd-5b914b91297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:10.046779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dab1f6-0782-4122-9cbd-5b914b91297a admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:10.047067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dab1f6-0782-4122-9cbd-5b914b91297a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:10.047345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dab1f6-0782-4122-9cbd-5b914b91297a admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:10.048133 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dab1f6-0782-4122-9cbd-5b914b91297a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:10.049770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dab1f6-0782-4122-9cbd-5b914b91297a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:10.089613 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 875/1752] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:10.090518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dab1f6-0782-4122-9cbd-5b914b91297a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:10.148354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dab1f6-0782-4122-9cbd-5b914b91297a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:10.176420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-dee8bc3b-fc2d-42f7-b1be-7fff9b9019a0 None None] MFA Rules not processed for user `a37f5d5ddd9845fbaf7d5645c31b2d40`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:10.181700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6372254-9b06-425b-9890-08045910a7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:10.182346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-dee8bc3b-fc2d-42f7-b1be-7fff9b9019a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:10.182840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6372254-9b06-425b-9890-08045910a7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:10.183267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6372254-9b06-425b-9890-08045910a7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:10.184485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6372254-9b06-425b-9890-08045910a7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:10.215032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6372254-9b06-425b-9890-08045910a7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:10.228745 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6372254-9b06-425b-9890-08045910a7d2 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:10.257346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 876/1753] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:10.260682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6372254-9b06-425b-9890-08045910a7d2 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:10.293338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6372254-9b06-425b-9890-08045910a7d2 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:10.320790 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f4f6a-1492-4d6a-ac22-b43e75289628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:10.322617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f4f6a-1492-4d6a-ac22-b43e75289628 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:10.323317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f4f6a-1492-4d6a-ac22-b43e75289628 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:10.323923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f4f6a-1492-4d6a-ac22-b43e75289628 admin admin] PATH_INFO: `/v3/projects/4e84895304ef46cd92ff838c34ccafa7/users/053d8ab14ec1455a8672142eeb1d1a4f/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:10.384057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f4f6a-1492-4d6a-ac22-b43e75289628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e84895304ef46cd92ff838c34ccafa7, user_id=053d8ab14ec1455a8672142eeb1d1a4f, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:10.395740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f4f6a-1492-4d6a-ac22-b43e75289628 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:10.429345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 877/1754] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:10 2022] PUT /identity/v3/projects/4e84895304ef46cd92ff838c34ccafa7/users/053d8ab14ec1455a8672142eeb1d1a4f/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:10.433206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f4f6a-1492-4d6a-ac22-b43e75289628 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:10.548797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 878/1755] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:56:10.571268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8bc3b-fc2d-42f7-b1be-7fff9b9019a0 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:10.573611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-dee8bc3b-fc2d-42f7-b1be-7fff9b9019a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:10.594194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f4f6a-1492-4d6a-ac22-b43e75289628 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:10.618234 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232c064-031c-4b31-8603-107c67e103a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:10.620325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232c064-031c-4b31-8603-107c67e103a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:10.620573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232c064-031c-4b31-8603-107c67e103a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:10.622429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232c064-031c-4b31-8603-107c67e103a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:10.633869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232c064-031c-4b31-8603-107c67e103a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:10.636222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232c064-031c-4b31-8603-107c67e103a5 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:10.683944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 878/1756] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:10.685032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232c064-031c-4b31-8603-107c67e103a5 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:10.716642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232c064-031c-4b31-8603-107c67e103a5 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:10.740772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9101-d6e1-4a38-a9d9-f81751fb782d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:10.741715 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9101-d6e1-4a38-a9d9-f81751fb782d admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:10.742055 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9101-d6e1-4a38-a9d9-f81751fb782d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:10.742421 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9101-d6e1-4a38-a9d9-f81751fb782d admin admin] PATH_INFO: `/v3/projects/4e84895304ef46cd92ff838c34ccafa7/users/053d8ab14ec1455a8672142eeb1d1a4f/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:10.744483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9101-d6e1-4a38-a9d9-f81751fb782d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e84895304ef46cd92ff838c34ccafa7, user_id=053d8ab14ec1455a8672142eeb1d1a4f, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:10.747870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9101-d6e1-4a38-a9d9-f81751fb782d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:10.798976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 879/1757] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:10 2022] PUT /identity/v3/projects/4e84895304ef46cd92ff838c34ccafa7/users/053d8ab14ec1455a8672142eeb1d1a4f/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:10.810544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f863ec0-3257-4d72-a662-ed856a0a0988 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:10.811075 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f863ec0-3257-4d72-a662-ed856a0a0988 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:10.812458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f863ec0-3257-4d72-a662-ed856a0a0988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:10.827877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8bc3b-fc2d-42f7-b1be-7fff9b9019a0 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:10.933615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-7f863ec0-3257-4d72-a662-ed856a0a0988 None None] MFA Rules not processed for user `053d8ab14ec1455a8672142eeb1d1a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:10.936869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7f863ec0-3257-4d72-a662-ed856a0a0988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:11.026568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b5e24-ff66-4f9b-beba-a58c7d57ddd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:11.028098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b5e24-ff66-4f9b-beba-a58c7d57ddd0 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:11.028615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b5e24-ff66-4f9b-beba-a58c7d57ddd0 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:11.029044 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b5e24-ff66-4f9b-beba-a58c7d57ddd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:11.152802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b5e24-ff66-4f9b-beba-a58c7d57ddd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:11.155259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b5e24-ff66-4f9b-beba-a58c7d57ddd0 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:11.173364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 880/1758] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:10 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:56:11.174528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 879/1759] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:56:11.187838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f863ec0-3257-4d72-a662-ed856a0a0988 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:11.190600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7f863ec0-3257-4d72-a662-ed856a0a0988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:11.318130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f863ec0-3257-4d72-a662-ed856a0a0988 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:11.323850 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67886248-d6ac-4aab-996c-2e941158dace service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:11.325122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67886248-d6ac-4aab-996c-2e941158dace service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:11.325796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67886248-d6ac-4aab-996c-2e941158dace service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:11.326117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67886248-d6ac-4aab-996c-2e941158dace service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:11.341763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67886248-d6ac-4aab-996c-2e941158dace service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:11.343563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67886248-d6ac-4aab-996c-2e941158dace service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:11.359087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 881/1760] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:11 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:11.564797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b5e24-ff66-4f9b-beba-a58c7d57ddd0 service nova] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:11.590119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b5e24-ff66-4f9b-beba-a58c7d57ddd0 service nova] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:11.609804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15db464-9319-4acf-85be-e0a49ec35440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:11.610658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15db464-9319-4acf-85be-e0a49ec35440 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:11.611081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15db464-9319-4acf-85be-e0a49ec35440 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:11.611465 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15db464-9319-4acf-85be-e0a49ec35440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:11.747996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15db464-9319-4acf-85be-e0a49ec35440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:11.749748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15db464-9319-4acf-85be-e0a49ec35440 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:11.774383 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 880/1761] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:11.882068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67886248-d6ac-4aab-996c-2e941158dace service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:11.884971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-67886248-d6ac-4aab-996c-2e941158dace service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:12.175632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67886248-d6ac-4aab-996c-2e941158dace service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:12.198914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:12.203318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:12.204993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:12.205152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] PATH_INFO: `/v3/projects/631ba39d9ea44f5eaefac6a44deac667` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:12.208363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=631ba39d9ea44f5eaefac6a44deac667)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:12.213163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:12.403768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '631ba39d9ea44f5eaefac6a44deac667', 'request_id': 'req-be4d0a31-1277-403d-a1c3-72559363a8f2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:12.431816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '631ba39d9ea44f5eaefac6a44deac667', 'request_id': 'req-be4d0a31-1277-403d-a1c3-72559363a8f2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:12.471599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '631ba39d9ea44f5eaefac6a44deac667', 'request_id': 'req-be4d0a31-1277-403d-a1c3-72559363a8f2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:12.535145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] The token cache is being invalidate because project 631ba39d9ea44f5eaefac6a44deac667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:12.542702 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:12.545551 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be4d0a31-1277-403d-a1c3-72559363a8f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:12.557823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 882/1762] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:11 2022] DELETE /identity/v3/projects/631ba39d9ea44f5eaefac6a44deac667 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:12.559369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15db464-9319-4acf-85be-e0a49ec35440 service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:12.576426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15db464-9319-4acf-85be-e0a49ec35440 service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:12.799736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:12.800604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:12.800893 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:12.801189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] PATH_INFO: `/v3/users/f6a5cae3bece4c6e9f48d3b2c5907545` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:12.804141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a5cae3bece4c6e9f48d3b2c5907545)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:12.805766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:12.898997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455e5a6-4e96-4586-a65f-fca95a39e51e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:12.901759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455e5a6-4e96-4586-a65f-fca95a39e51e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:12.902541 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455e5a6-4e96-4586-a65f-fca95a39e51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:12.935342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a5cae3bece4c6e9f48d3b2c5907545'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:12.941433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a5cae3bece4c6e9f48d3b2c5907545'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:12.973741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a5cae3bece4c6e9f48d3b2c5907545'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:12.978240 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a5cae3bece4c6e9f48d3b2c5907545'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:13.002992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a5cae3bece4c6e9f48d3b2c5907545'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:13.007173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-7455e5a6-4e96-4586-a65f-fca95a39e51e None None] MFA Rules not processed for user `648ae92126944b50a22ad780e92df914`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:13.014962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7455e5a6-4e96-4586-a65f-fca95a39e51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:13.040638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 881/1763] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:56:12 2022] DELETE /identity/v3/users/f6a5cae3bece4c6e9f48d3b2c5907545 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:56:13.205295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 883/1764] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:56:14.627828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:14.632303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:14.968302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2d2a8-3a4f-4487-985d-d641cdad5003 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:14.995017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:14.996808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:14.997537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:14.998232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] PATH_INFO: `/v3/projects/cc9d3d16e75342b1a000f597fead9f04` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:15.019301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc9d3d16e75342b1a000f597fead9f04)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:15.022493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:15.180572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc9d3d16e75342b1a000f597fead9f04', 'request_id': 'req-cdc5b47c-4432-4437-886e-447069e984f0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:15.203868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc9d3d16e75342b1a000f597fead9f04', 'request_id': 'req-cdc5b47c-4432-4437-886e-447069e984f0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:15.210440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc9d3d16e75342b1a000f597fead9f04', 'request_id': 'req-cdc5b47c-4432-4437-886e-447069e984f0'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:15.239428 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] The token cache is being invalidate because project cc9d3d16e75342b1a000f597fead9f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:15.239830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:15.240531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-cdc5b47c-4432-4437-886e-447069e984f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:15.245711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 882/1765] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:14 2022] DELETE /identity/v3/projects/cc9d3d16e75342b1a000f597fead9f04 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:15.248320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68978c06-2b06-4fc2-a5f6-93ec764b21ad None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:15.249560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68978c06-2b06-4fc2-a5f6-93ec764b21ad None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:15.250025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68978c06-2b06-4fc2-a5f6-93ec764b21ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:15.298021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-68978c06-2b06-4fc2-a5f6-93ec764b21ad None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:15.304687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-68978c06-2b06-4fc2-a5f6-93ec764b21ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:15.521534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 884/1766] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:56:15.531275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033d53c-15ba-43f0-9cfd-5373235c3c24 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:15.532128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033d53c-15ba-43f0-9cfd-5373235c3c24 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:15.533565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033d53c-15ba-43f0-9cfd-5373235c3c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:15.591262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-8033d53c-15ba-43f0-9cfd-5373235c3c24 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:15.592479 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8033d53c-15ba-43f0-9cfd-5373235c3c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:15.632959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 883/1767] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:15.634293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68978c06-2b06-4fc2-a5f6-93ec764b21ad None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:15.646098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68978c06-2b06-4fc2-a5f6-93ec764b21ad None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:15.668822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd35fd-7ead-42bf-85ac-55944d902a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:15.670104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd35fd-7ead-42bf-85ac-55944d902a7a admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:15.670783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd35fd-7ead-42bf-85ac-55944d902a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:15.671321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd35fd-7ead-42bf-85ac-55944d902a7a admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:15.673516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd35fd-7ead-42bf-85ac-55944d902a7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:56:15.674093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd35fd-7ead-42bf-85ac-55944d902a7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:15.684089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd35fd-7ead-42bf-85ac-55944d902a7a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:15.740187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 885/1768] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:56:15.744733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033d53c-15ba-43f0-9cfd-5373235c3c24 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:15.757489 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd35fd-7ead-42bf-85ac-55944d902a7a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:15.761036 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033d53c-15ba-43f0-9cfd-5373235c3c24 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:15.768024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ccfd35fd-7ead-42bf-85ac-55944d902a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:15.792499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765909-1c45-488d-8f1e-9746aa38f68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:15.793227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765909-1c45-488d-8f1e-9746aa38f68d admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:15.793536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765909-1c45-488d-8f1e-9746aa38f68d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:15.793831 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765909-1c45-488d-8f1e-9746aa38f68d admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:15.794565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765909-1c45-488d-8f1e-9746aa38f68d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:15.801732 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765909-1c45-488d-8f1e-9746aa38f68d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:15.810784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd35fd-7ead-42bf-85ac-55944d902a7a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:15.848850 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:15.857101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:15.862919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:15.863542 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] PATH_INFO: `/v3/projects/8e4d495b764f4be0aa6535a9f3fbdf08` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:15.870884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e4d495b764f4be0aa6535a9f3fbdf08)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:15.883206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:15.949419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 884/1769] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:56:15 2022] POST /identity/v3/projects => generated 371 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:15.955709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765909-1c45-488d-8f1e-9746aa38f68d admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:16.110333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e4d495b764f4be0aa6535a9f3fbdf08', 'request_id': 'req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:16.139182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e4d495b764f4be0aa6535a9f3fbdf08', 'request_id': 'req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:16.183787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e4d495b764f4be0aa6535a9f3fbdf08', 'request_id': 'req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:16.193647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765909-1c45-488d-8f1e-9746aa38f68d admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:16.236918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] The token cache is being invalidate because project 8e4d495b764f4be0aa6535a9f3fbdf08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:16.237811 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:16.239313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-51ddbd0e-63f2-434a-9b6c-05d1d6d63a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:16.246228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 886/1770] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:15 2022] DELETE /identity/v3/projects/8e4d495b764f4be0aa6535a9f3fbdf08 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:16.246583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89997-b37e-4763-b9f0-c243f1b68c3a None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:16.248003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89997-b37e-4763-b9f0-c243f1b68c3a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:16.248117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89997-b37e-4763-b9f0-c243f1b68c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:16.351827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-79c89997-b37e-4763-b9f0-c243f1b68c3a None None] MFA Rules not processed for user `8b2980196e0f4fafa5131dd5de02ac75`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:16.353222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-79c89997-b37e-4763-b9f0-c243f1b68c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:16.427272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45f972-bd3a-40a6-9880-ee8ce71b62e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:16.429043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45f972-bd3a-40a6-9880-ee8ce71b62e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:16.429467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45f972-bd3a-40a6-9880-ee8ce71b62e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:16.430051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45f972-bd3a-40a6-9880-ee8ce71b62e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:16.433591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45f972-bd3a-40a6-9880-ee8ce71b62e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:16.436655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45f972-bd3a-40a6-9880-ee8ce71b62e2 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:16.498133 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 885/1771] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:56:15 2022] POST /identity/v3/users => generated 286 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:16.503227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45f972-bd3a-40a6-9880-ee8ce71b62e2 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:16.510244 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1d45f972-bd3a-40a6-9880-ee8ce71b62e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:16.605060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 887/1772] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:16.613277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c89997-b37e-4763-b9f0-c243f1b68c3a None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:16.617881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45f972-bd3a-40a6-9880-ee8ce71b62e2 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:16.627186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-79c89997-b37e-4763-b9f0-c243f1b68c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:16.655607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:16.664556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:16.665877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:16.666962 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] PATH_INFO: `/v3/users/710739820a0042be80340adb275d0d1c` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:16.675557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=710739820a0042be80340adb275d0d1c)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:16.689900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:16.690952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c89997-b37e-4763-b9f0-c243f1b68c3a None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:16.713140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a5260-f0a0-4ae7-8807-a82569d190c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:16.714004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a5260-f0a0-4ae7-8807-a82569d190c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:16.714473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a5260-f0a0-4ae7-8807-a82569d190c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:16.714943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a5260-f0a0-4ae7-8807-a82569d190c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:16.715939 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a5260-f0a0-4ae7-8807-a82569d190c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:16.719208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a5260-f0a0-4ae7-8807-a82569d190c9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:16.766152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 888/1773] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:16.766152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a5260-f0a0-4ae7-8807-a82569d190c9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:16.777264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-725a5260-f0a0-4ae7-8807-a82569d190c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:16.875241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '710739820a0042be80340adb275d0d1c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:16.876756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '710739820a0042be80340adb275d0d1c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:16.915339 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '710739820a0042be80340adb275d0d1c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:16.916084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '710739820a0042be80340adb275d0d1c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:16.925126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '710739820a0042be80340adb275d0d1c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:16.960136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 886/1774] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:56:16 2022] DELETE /identity/v3/users/710739820a0042be80340adb275d0d1c => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:16.960136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:16.964544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:17.064245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a5260-f0a0-4ae7-8807-a82569d190c9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:17.194033 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a764934-a124-411e-8735-855f6023c541 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:17.207853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799c997-7545-402f-9dd2-f7adaec63639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:17.211546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799c997-7545-402f-9dd2-f7adaec63639 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:17.211782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799c997-7545-402f-9dd2-f7adaec63639 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:17.212194 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799c997-7545-402f-9dd2-f7adaec63639 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:17.213226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799c997-7545-402f-9dd2-f7adaec63639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:17.217302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799c997-7545-402f-9dd2-f7adaec63639 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:17.267441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 887/1775] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:17.275829 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799c997-7545-402f-9dd2-f7adaec63639 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:17.338910 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2142925-d36f-4c35-af96-e69eb06ba2e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:17.339737 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2142925-d36f-4c35-af96-e69eb06ba2e3 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:17.340391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2142925-d36f-4c35-af96-e69eb06ba2e3 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:17.340897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2142925-d36f-4c35-af96-e69eb06ba2e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:17.343534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a2142925-d36f-4c35-af96-e69eb06ba2e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:17.385655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799c997-7545-402f-9dd2-f7adaec63639 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:17.419591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282744-c633-43ad-9b11-09ec9f7a9751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:17.425490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282744-c633-43ad-9b11-09ec9f7a9751 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:17.427350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282744-c633-43ad-9b11-09ec9f7a9751 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:17.427618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282744-c633-43ad-9b11-09ec9f7a9751 admin admin] PATH_INFO: `/v3/projects/703aa4774820401da920e420eb6181d9/users/8ef0eaa76ffe4b62a132f0f4ee595a6f/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:17.494534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d282744-c633-43ad-9b11-09ec9f7a9751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=703aa4774820401da920e420eb6181d9, user_id=8ef0eaa76ffe4b62a132f0f4ee595a6f, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:17.503543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d282744-c633-43ad-9b11-09ec9f7a9751 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:17.515718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2142925-d36f-4c35-af96-e69eb06ba2e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:17.517884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2142925-d36f-4c35-af96-e69eb06ba2e3 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:17.530484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 889/1776] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:17.551924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027697ac-4ff3-4723-b588-a57ddce3cd2c None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:17.552342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027697ac-4ff3-4723-b588-a57ddce3cd2c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:17.552718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027697ac-4ff3-4723-b588-a57ddce3cd2c None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:17.554826 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 890/1777] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 13:56:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:56:17.567163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 888/1778] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:17 2022] PUT /identity/v3/projects/703aa4774820401da920e420eb6181d9/users/8ef0eaa76ffe4b62a132f0f4ee595a6f/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:17.569617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbcd4e-ad84-4598-b770-1833f54d1ec6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:17.569967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbcd4e-ad84-4598-b770-1833f54d1ec6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:17.570246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbcd4e-ad84-4598-b770-1833f54d1ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:17.611864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-abfbcd4e-ad84-4598-b770-1833f54d1ec6 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:17.615775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-abfbcd4e-ad84-4598-b770-1833f54d1ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:17.744655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 891/1779] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:17.831748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ab94b-8776-40e9-8e34-f1a63273ddf2 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:17.832336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ab94b-8776-40e9-8e34-f1a63273ddf2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:17.832336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ab94b-8776-40e9-8e34-f1a63273ddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:17.898699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a38ab94b-8776-40e9-8e34-f1a63273ddf2 None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:17.900373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a38ab94b-8776-40e9-8e34-f1a63273ddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:18.092940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 889/1780] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 13:56:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:18.133146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbcd4e-ad84-4598-b770-1833f54d1ec6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:18.135040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-abfbcd4e-ad84-4598-b770-1833f54d1ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:18.204611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ab94b-8776-40e9-8e34-f1a63273ddf2 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:18.212315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a38ab94b-8776-40e9-8e34-f1a63273ddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:18.257149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ab94b-8776-40e9-8e34-f1a63273ddf2 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:18.281844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47535c6d-6fa3-4ee5-b96a-c1498c9a66be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:18.282540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47535c6d-6fa3-4ee5-b96a-c1498c9a66be service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:18.282839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47535c6d-6fa3-4ee5-b96a-c1498c9a66be service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:18.283232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47535c6d-6fa3-4ee5-b96a-c1498c9a66be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:18.325802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47535c6d-6fa3-4ee5-b96a-c1498c9a66be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:18.335680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47535c6d-6fa3-4ee5-b96a-c1498c9a66be service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:18.413045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbcd4e-ad84-4598-b770-1833f54d1ec6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:18.433473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 890/1781] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:18.461659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6b9d9-c2f4-4384-a074-4548e579be43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:18.465366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6b9d9-c2f4-4384-a074-4548e579be43 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:18.467838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6b9d9-c2f4-4384-a074-4548e579be43 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:18.468140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6b9d9-c2f4-4384-a074-4548e579be43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:18.533249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6b9d9-c2f4-4384-a074-4548e579be43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:18.540823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6b9d9-c2f4-4384-a074-4548e579be43 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:18.597815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 892/1782] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:18.906047 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47535c6d-6fa3-4ee5-b96a-c1498c9a66be service nova] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:18.908229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-47535c6d-6fa3-4ee5-b96a-c1498c9a66be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:18.979827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47535c6d-6fa3-4ee5-b96a-c1498c9a66be service nova] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:19.008790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:19.013623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:19.016556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:19.018854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] PATH_INFO: `/v3/projects/a40e41e735744f7592bd07272829295d` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:19.025679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a40e41e735744f7592bd07272829295d)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:19.029221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:19.167487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a40e41e735744f7592bd07272829295d', 'request_id': 'req-c35ee472-6286-4db9-a003-2b983e44c5f5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:19.195407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a40e41e735744f7592bd07272829295d', 'request_id': 'req-c35ee472-6286-4db9-a003-2b983e44c5f5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:19.212223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a40e41e735744f7592bd07272829295d', 'request_id': 'req-c35ee472-6286-4db9-a003-2b983e44c5f5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:19.233879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] The token cache is being invalidate because project a40e41e735744f7592bd07272829295d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:19.234080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:19.235489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c35ee472-6286-4db9-a003-2b983e44c5f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:19.244934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 891/1783] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:18 2022] DELETE /identity/v3/projects/a40e41e735744f7592bd07272829295d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:19.251273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8238cdc-b5b0-4756-bb85-b05093da648f None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:19.251557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8238cdc-b5b0-4756-bb85-b05093da648f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:19.251839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8238cdc-b5b0-4756-bb85-b05093da648f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:19.305250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f8238cdc-b5b0-4756-bb85-b05093da648f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:19.310414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f8238cdc-b5b0-4756-bb85-b05093da648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:19.514456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 893/1784] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:19.524132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad89a03-3e6d-4afa-a6cc-325c9358891f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:19.524427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad89a03-3e6d-4afa-a6cc-325c9358891f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:19.524704 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad89a03-3e6d-4afa-a6cc-325c9358891f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:19.571819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-9ad89a03-3e6d-4afa-a6cc-325c9358891f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:19.575845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9ad89a03-3e6d-4afa-a6cc-325c9358891f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:19.610613 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 892/1785] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:19.612055 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8238cdc-b5b0-4756-bb85-b05093da648f None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:19.641350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8238cdc-b5b0-4756-bb85-b05093da648f None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:19.675650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9fc1c-ddc8-4ec1-a053-0dbe30d4976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:19.676363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9fc1c-ddc8-4ec1-a053-0dbe30d4976b admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:19.676646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9fc1c-ddc8-4ec1-a053-0dbe30d4976b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:19.676919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9fc1c-ddc8-4ec1-a053-0dbe30d4976b admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:19.677824 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9fc1c-ddc8-4ec1-a053-0dbe30d4976b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:56:19.678134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9fc1c-ddc8-4ec1-a053-0dbe30d4976b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:19.679859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9fc1c-ddc8-4ec1-a053-0dbe30d4976b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:19.724005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 894/1786] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:56:19.729797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad89a03-3e6d-4afa-a6cc-325c9358891f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:19.759333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad89a03-3e6d-4afa-a6cc-325c9358891f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:19.772540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735df42-2aba-4d60-978b-7ba3a3fd5b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:19.773587 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735df42-2aba-4d60-978b-7ba3a3fd5b47 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:19.774024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735df42-2aba-4d60-978b-7ba3a3fd5b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:19.774469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735df42-2aba-4d60-978b-7ba3a3fd5b47 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:19.775555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735df42-2aba-4d60-978b-7ba3a3fd5b47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:19.778105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735df42-2aba-4d60-978b-7ba3a3fd5b47 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:19.862697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9fc1c-ddc8-4ec1-a053-0dbe30d4976b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:19.872157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 893/1787] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:56:19 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:19.877762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9fc1c-ddc8-4ec1-a053-0dbe30d4976b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:19.919518 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd07216-2ba9-4531-9217-b925a29abd82 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:19.920237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd07216-2ba9-4531-9217-b925a29abd82 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:19.920320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd07216-2ba9-4531-9217-b925a29abd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:20.003430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-2cd07216-2ba9-4531-9217-b925a29abd82 None None] MFA Rules not processed for user `20fd230cc30f4980bedebc75c620d707`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:20.008382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2cd07216-2ba9-4531-9217-b925a29abd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:20.135123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe0533-836a-47cc-9c8b-df366392d869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:20.138499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe0533-836a-47cc-9c8b-df366392d869 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:20.139567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe0533-836a-47cc-9c8b-df366392d869 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:20.144008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe0533-836a-47cc-9c8b-df366392d869 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:20.144911 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe0533-836a-47cc-9c8b-df366392d869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:20.147157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe0533-836a-47cc-9c8b-df366392d869 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:20.227222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 895/1788] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:56:19 2022] POST /identity/v3/users => generated 294 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:20.229099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe0533-836a-47cc-9c8b-df366392d869 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:20.266971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe0533-836a-47cc-9c8b-df366392d869 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:20.285525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ceaebf-338d-438e-b5f7-0d4c77184a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:20.286319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ceaebf-338d-438e-b5f7-0d4c77184a87 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:20.286627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ceaebf-338d-438e-b5f7-0d4c77184a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:20.289169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ceaebf-338d-438e-b5f7-0d4c77184a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:20.291109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ceaebf-338d-438e-b5f7-0d4c77184a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:20.293822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ceaebf-338d-438e-b5f7-0d4c77184a87 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:20.310239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 894/1789] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:20.312197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a9d82-04b6-47ae-a86a-03384ebcde3e None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:20.312490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a9d82-04b6-47ae-a86a-03384ebcde3e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:20.313503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a9d82-04b6-47ae-a86a-03384ebcde3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:20.326030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 896/1790] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:20.336066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ceaebf-338d-438e-b5f7-0d4c77184a87 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:20.359363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7f4a9d82-04b6-47ae-a86a-03384ebcde3e None None] MFA Rules not processed for user `20fd230cc30f4980bedebc75c620d707`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:20.366986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7f4a9d82-04b6-47ae-a86a-03384ebcde3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:20.377375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ceaebf-338d-438e-b5f7-0d4c77184a87 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:20.414409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 895/1791] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:20.418807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e9335-edc1-4b5f-b620-510534a04a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:20.423480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e9335-edc1-4b5f-b620-510534a04a28 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:20.424152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e9335-edc1-4b5f-b620-510534a04a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:20.428239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e9335-edc1-4b5f-b620-510534a04a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:20.429153 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e9335-edc1-4b5f-b620-510534a04a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:20.433598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e9335-edc1-4b5f-b620-510534a04a28 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:20.462139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 897/1792] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:20.469275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a9d82-04b6-47ae-a86a-03384ebcde3e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:20.502645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a9d82-04b6-47ae-a86a-03384ebcde3e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:20.508663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa2b72-57e5-46bc-b9e5-d24c10cd4012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:20.509879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2b72-57e5-46bc-b9e5-d24c10cd4012 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:20.518879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2b72-57e5-46bc-b9e5-d24c10cd4012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:20.519754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2b72-57e5-46bc-b9e5-d24c10cd4012 admin admin] PATH_INFO: `/v3/projects/b09b8a26e3364d17b733d9d3c3181892/users/2c724201e0874436aeeb8926925012d4/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:20.568849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa2b72-57e5-46bc-b9e5-d24c10cd4012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b09b8a26e3364d17b733d9d3c3181892, user_id=2c724201e0874436aeeb8926925012d4, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:20.576027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa2b72-57e5-46bc-b9e5-d24c10cd4012 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:20.631631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 896/1793] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:20 2022] PUT /identity/v3/projects/b09b8a26e3364d17b733d9d3c3181892/users/2c724201e0874436aeeb8926925012d4/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:20.645856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf9e8f-573e-42f0-b2db-3cd5cf259949 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:20.646513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf9e8f-573e-42f0-b2db-3cd5cf259949 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:20.647771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf9e8f-573e-42f0-b2db-3cd5cf259949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:20.722221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f6cf9e8f-573e-42f0-b2db-3cd5cf259949 None None] MFA Rules not processed for user `2c724201e0874436aeeb8926925012d4`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:20.726645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f6cf9e8f-573e-42f0-b2db-3cd5cf259949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:20.849873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa2b72-57e5-46bc-b9e5-d24c10cd4012 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:20.855272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f5fa2b72-57e5-46bc-b9e5-d24c10cd4012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:20.962984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 898/1794] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:20.964009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf9e8f-573e-42f0-b2db-3cd5cf259949 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:21.030065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa2b72-57e5-46bc-b9e5-d24c10cd4012 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:21.042170 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17241b1-3401-4216-8f53-13a890d1f5cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:21.043887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17241b1-3401-4216-8f53-13a890d1f5cf service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:21.044605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17241b1-3401-4216-8f53-13a890d1f5cf service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:21.045241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17241b1-3401-4216-8f53-13a890d1f5cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:21.127307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf9e8f-573e-42f0-b2db-3cd5cf259949 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:21.136347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5ff11-e304-40ba-8eb6-bbe761612f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:21.136648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5ff11-e304-40ba-8eb6-bbe761612f03 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:21.136923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5ff11-e304-40ba-8eb6-bbe761612f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:21.137276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5ff11-e304-40ba-8eb6-bbe761612f03 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:21.139792 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5ff11-e304-40ba-8eb6-bbe761612f03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:21.143488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5ff11-e304-40ba-8eb6-bbe761612f03 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:21.197379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17241b1-3401-4216-8f53-13a890d1f5cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:21.199045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17241b1-3401-4216-8f53-13a890d1f5cf service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:21.229684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 899/1795] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:56:20 2022] POST /identity/v3/projects => generated 385 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:21.233603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 897/1796] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:21.236567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5ff11-e304-40ba-8eb6-bbe761612f03 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:21.462662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5ff11-e304-40ba-8eb6-bbe761612f03 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:21.470879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef6883-f22f-45d0-ae37-850494a47da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:21.472190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef6883-f22f-45d0-ae37-850494a47da4 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:21.472190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef6883-f22f-45d0-ae37-850494a47da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:21.472659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef6883-f22f-45d0-ae37-850494a47da4 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:21.473582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef6883-f22f-45d0-ae37-850494a47da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:21.477447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef6883-f22f-45d0-ae37-850494a47da4 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:21.511124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 900/1797] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:56:21 2022] POST /identity/v3/users => generated 299 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:21.512811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17241b1-3401-4216-8f53-13a890d1f5cf service nova] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:21.532910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17241b1-3401-4216-8f53-13a890d1f5cf service nova] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:21.556171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583643b-3c4c-423e-bb17-27079e947405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:21.557342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583643b-3c4c-423e-bb17-27079e947405 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:21.557793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583643b-3c4c-423e-bb17-27079e947405 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:21.558522 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583643b-3c4c-423e-bb17-27079e947405 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:21.559870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583643b-3c4c-423e-bb17-27079e947405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:21.571528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583643b-3c4c-423e-bb17-27079e947405 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:21.610405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 898/1798] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:21.619737 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef6883-f22f-45d0-ae37-850494a47da4 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:21.714955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef6883-f22f-45d0-ae37-850494a47da4 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:21.759249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a530c6-f8a1-4e17-854c-c9d5cea42bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:21.764322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a530c6-f8a1-4e17-854c-c9d5cea42bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:21.766927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a530c6-f8a1-4e17-854c-c9d5cea42bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:21.767551 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a530c6-f8a1-4e17-854c-c9d5cea42bc0 admin admin] PATH_INFO: `/v3/projects/a0a7556849be4e80a1e56245a04d0749/users/6f2eb69ea2ce4a25a9a0dc1fd78913c6/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:21.832294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a530c6-f8a1-4e17-854c-c9d5cea42bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a7556849be4e80a1e56245a04d0749, user_id=6f2eb69ea2ce4a25a9a0dc1fd78913c6, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:21.853733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a530c6-f8a1-4e17-854c-c9d5cea42bc0 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:21.909670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 901/1799] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:21 2022] PUT /identity/v3/projects/a0a7556849be4e80a1e56245a04d0749/users/6f2eb69ea2ce4a25a9a0dc1fd78913c6/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:21.912286 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583643b-3c4c-423e-bb17-27079e947405 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:21.951156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583643b-3c4c-423e-bb17-27079e947405 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:22.202649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c895b1-60f0-43c6-b088-d348ae4de225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:22.203658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c895b1-60f0-43c6-b088-d348ae4de225 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:22.204434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c895b1-60f0-43c6-b088-d348ae4de225 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:22.209506 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c895b1-60f0-43c6-b088-d348ae4de225 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:22.211195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c895b1-60f0-43c6-b088-d348ae4de225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:22.213977 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c895b1-60f0-43c6-b088-d348ae4de225 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:22.259711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a530c6-f8a1-4e17-854c-c9d5cea42bc0 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:22.263440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 899/1800] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:22.294742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a530c6-f8a1-4e17-854c-c9d5cea42bc0 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:22.325601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce584e-c63f-465e-a160-999fa1612a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:22.327229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce584e-c63f-465e-a160-999fa1612a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:22.327718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce584e-c63f-465e-a160-999fa1612a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:22.328246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce584e-c63f-465e-a160-999fa1612a29 admin admin] PATH_INFO: `/v3/projects/a0a7556849be4e80a1e56245a04d0749/users/6f2eb69ea2ce4a25a9a0dc1fd78913c6/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:22.349834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce584e-c63f-465e-a160-999fa1612a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a7556849be4e80a1e56245a04d0749, user_id=6f2eb69ea2ce4a25a9a0dc1fd78913c6, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:22.362009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce584e-c63f-465e-a160-999fa1612a29 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:22.416244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 902/1801] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:22 2022] PUT /identity/v3/projects/a0a7556849be4e80a1e56245a04d0749/users/6f2eb69ea2ce4a25a9a0dc1fd78913c6/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:22.418420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4765fa3-2bf4-4e4f-955b-ed0c5a58e9da None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:22.418898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4765fa3-2bf4-4e4f-955b-ed0c5a58e9da None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:22.419344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4765fa3-2bf4-4e4f-955b-ed0c5a58e9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:22.472894 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e4765fa3-2bf4-4e4f-955b-ed0c5a58e9da None None] MFA Rules not processed for user `6f2eb69ea2ce4a25a9a0dc1fd78913c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:22.474368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e4765fa3-2bf4-4e4f-955b-ed0c5a58e9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:22.637035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 900/1802] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:56:22.651035 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce584e-c63f-465e-a160-999fa1612a29 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:22.652877 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c0ce584e-c63f-465e-a160-999fa1612a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:22.778735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce584e-c63f-465e-a160-999fa1612a29 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:22.792736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb71b72-43b3-41be-8e32-b2d47794f41b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:22.793810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb71b72-43b3-41be-8e32-b2d47794f41b service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:22.794254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb71b72-43b3-41be-8e32-b2d47794f41b service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:22.794679 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb71b72-43b3-41be-8e32-b2d47794f41b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:22.886589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb71b72-43b3-41be-8e32-b2d47794f41b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:22.889556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb71b72-43b3-41be-8e32-b2d47794f41b service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:22.949470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 903/1803] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:23.665600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479330f-434d-4689-8edd-28673ca3340f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:23.667532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479330f-434d-4689-8edd-28673ca3340f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:23.668173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479330f-434d-4689-8edd-28673ca3340f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:23.710090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-8479330f-434d-4689-8edd-28673ca3340f None None] MFA Rules not processed for user `a37f5d5ddd9845fbaf7d5645c31b2d40`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:23.713186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8479330f-434d-4689-8edd-28673ca3340f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:23.918321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 901/1804] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:24.354569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb71b72-43b3-41be-8e32-b2d47794f41b service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:24.462573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb71b72-43b3-41be-8e32-b2d47794f41b service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:24.476927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:24.477832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:24.478110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:24.478509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] PATH_INFO: `/v3/users/2c724201e0874436aeeb8926925012d4` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:24.480136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c724201e0874436aeeb8926925012d4)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:24.482427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:24.608324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c724201e0874436aeeb8926925012d4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:24.639925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c724201e0874436aeeb8926925012d4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:24.659709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c724201e0874436aeeb8926925012d4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:24.661038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c724201e0874436aeeb8926925012d4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:24.673338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-0080fad1-fbde-40e0-a40b-8cdb012d185e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c724201e0874436aeeb8926925012d4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:24.689290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 904/1805] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:56:24 2022] DELETE /identity/v3/users/2c724201e0874436aeeb8926925012d4 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:24.700586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca89ea-787e-4a08-af15-c96186179acd None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:24.701051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca89ea-787e-4a08-af15-c96186179acd None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:24.701140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca89ea-787e-4a08-af15-c96186179acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:24.732315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc366368-87df-49ef-9e41-bcfce645fcbd None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:24.732879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc366368-87df-49ef-9e41-bcfce645fcbd None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:24.734439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc366368-87df-49ef-9e41-bcfce645fcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:24.750921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-13ca89ea-787e-4a08-af15-c96186179acd None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:24.753123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-13ca89ea-787e-4a08-af15-c96186179acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:24.756027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-dc366368-87df-49ef-9e41-bcfce645fcbd None None] MFA Rules not processed for user `8b2980196e0f4fafa5131dd5de02ac75`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:24.757872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-dc366368-87df-49ef-9e41-bcfce645fcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:24.848360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 905/1806] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:24.880038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d7153-73a1-42ee-87b3-3eb6b3a8d2d6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:24.880386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d7153-73a1-42ee-87b3-3eb6b3a8d2d6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:24.880719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d7153-73a1-42ee-87b3-3eb6b3a8d2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:24.963321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 902/1807] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:24.990177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e59d7153-73a1-42ee-87b3-3eb6b3a8d2d6 None None] MFA Rules not processed for user `8ef0eaa76ffe4b62a132f0f4ee595a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:24.992710 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e59d7153-73a1-42ee-87b3-3eb6b3a8d2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:25.022336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca89ea-787e-4a08-af15-c96186179acd None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:25.025093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-13ca89ea-787e-4a08-af15-c96186179acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:25.300689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca89ea-787e-4a08-af15-c96186179acd None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:25.313702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934618f9-7a78-4f81-a204-86058375f7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:25.314386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934618f9-7a78-4f81-a204-86058375f7bc service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:25.314837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934618f9-7a78-4f81-a204-86058375f7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:25.315133 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934618f9-7a78-4f81-a204-86058375f7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:25.320785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 906/1808] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:25.334101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a315379-8f21-4283-b1a3-71f8845e148d None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:25.336484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a315379-8f21-4283-b1a3-71f8845e148d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:25.337360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a315379-8f21-4283-b1a3-71f8845e148d None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:25.354494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 907/1809] 10.222.0.79 () {54 vars in 912 bytes} [Fri May 13 13:56:25 2022] GET /identity => generated 270 bytes in 36 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:56:25.360447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb83b8-f5d1-48ed-9ba2-9d84d39c3251 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:25.361499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb83b8-f5d1-48ed-9ba2-9d84d39c3251 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:25.362798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb83b8-f5d1-48ed-9ba2-9d84d39c3251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:25.362951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934618f9-7a78-4f81-a204-86058375f7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:25.369683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934618f9-7a78-4f81-a204-86058375f7bc service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:25.390587 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 903/1810] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:25.395221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934618f9-7a78-4f81-a204-86058375f7bc service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:25.396678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-934618f9-7a78-4f81-a204-86058375f7bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:25.439853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-83bb83b8-f5d1-48ed-9ba2-9d84d39c3251 None None] MFA Rules not processed for user `8f784cc461ab48ad92140680758a3704`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:25.442100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-83bb83b8-f5d1-48ed-9ba2-9d84d39c3251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:25.473660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 908/1811] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 13:56:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:25.522247 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934618f9-7a78-4f81-a204-86058375f7bc service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:25.533697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb83b8-f5d1-48ed-9ba2-9d84d39c3251 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:25.534196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3a864-c0e3-429e-b6df-4ed82516286a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:25.534917 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3a864-c0e3-429e-b6df-4ed82516286a service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:25.535205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3a864-c0e3-429e-b6df-4ed82516286a service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:25.535486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3a864-c0e3-429e-b6df-4ed82516286a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:25.554854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3a864-c0e3-429e-b6df-4ed82516286a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:25.556356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3a864-c0e3-429e-b6df-4ed82516286a service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:25.562917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb83b8-f5d1-48ed-9ba2-9d84d39c3251 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:25.575528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4263c37-88fe-443a-94f6-309eba8b8ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:25.579030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 904/1812] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:25.579786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4263c37-88fe-443a-94f6-309eba8b8ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:25.580333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4263c37-88fe-443a-94f6-309eba8b8ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:25.580946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4263c37-88fe-443a-94f6-309eba8b8ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:25.595313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4263c37-88fe-443a-94f6-309eba8b8ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:25.597471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4263c37-88fe-443a-94f6-309eba8b8ba6 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:25.608986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 909/1813] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:27.125270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3a864-c0e3-429e-b6df-4ed82516286a service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:27.128327 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a8c3a864-c0e3-429e-b6df-4ed82516286a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:27.154279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3a864-c0e3-429e-b6df-4ed82516286a service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:27.162122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:27.164204 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:27.164760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:27.165115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] PATH_INFO: `/v3/projects/b09b8a26e3364d17b733d9d3c3181892` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:27.166822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b09b8a26e3364d17b733d9d3c3181892)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:27.185290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:27.275952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b09b8a26e3364d17b733d9d3c3181892', 'request_id': 'req-5921ff75-3573-4afc-8ef3-71e756eaa6b7'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:27.294418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b09b8a26e3364d17b733d9d3c3181892', 'request_id': 'req-5921ff75-3573-4afc-8ef3-71e756eaa6b7'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:27.315343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b09b8a26e3364d17b733d9d3c3181892', 'request_id': 'req-5921ff75-3573-4afc-8ef3-71e756eaa6b7'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:27.371651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] The token cache is being invalidate because project b09b8a26e3364d17b733d9d3c3181892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:27.372298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:27.375749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5921ff75-3573-4afc-8ef3-71e756eaa6b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:27.383570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 905/1814] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:27 2022] DELETE /identity/v3/projects/b09b8a26e3364d17b733d9d3c3181892 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:27.388015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4263c37-88fe-443a-94f6-309eba8b8ba6 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:27.410117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4263c37-88fe-443a-94f6-309eba8b8ba6 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:27.534879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:27.536042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:27.536335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:27.536619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] PATH_INFO: `/v3/users/6f2eb69ea2ce4a25a9a0dc1fd78913c6` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:27.538588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f2eb69ea2ce4a25a9a0dc1fd78913c6)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:27.540605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:27.670573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f2eb69ea2ce4a25a9a0dc1fd78913c6'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:27.712506 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f2eb69ea2ce4a25a9a0dc1fd78913c6'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:27.740293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f2eb69ea2ce4a25a9a0dc1fd78913c6'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:27.741931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f2eb69ea2ce4a25a9a0dc1fd78913c6'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:27.760553 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-8f25313b-2829-4cbc-9dde-6842b90207c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f2eb69ea2ce4a25a9a0dc1fd78913c6'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:27.780067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 910/1815] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:56:27 2022] DELETE /identity/v3/users/6f2eb69ea2ce4a25a9a0dc1fd78913c6 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:56:27.793879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622770ed-63ff-43bc-a98f-9fcf108cabef None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:27.794764 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622770ed-63ff-43bc-a98f-9fcf108cabef None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:27.795169 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622770ed-63ff-43bc-a98f-9fcf108cabef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:27.866171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-622770ed-63ff-43bc-a98f-9fcf108cabef None None] MFA Rules not processed for user `8ef0eaa76ffe4b62a132f0f4ee595a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:27.872172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-622770ed-63ff-43bc-a98f-9fcf108cabef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:28.056785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 906/1816] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:28.763940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d1a77-a3a4-4719-bc43-aaed084df671 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:28.764498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d1a77-a3a4-4719-bc43-aaed084df671 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:28.764744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d1a77-a3a4-4719-bc43-aaed084df671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:28.832661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-420d1a77-a3a4-4719-bc43-aaed084df671 None None] MFA Rules not processed for user `20fd230cc30f4980bedebc75c620d707`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:28.834322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-420d1a77-a3a4-4719-bc43-aaed084df671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:28.976661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 911/1817] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:29.076769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622770ed-63ff-43bc-a98f-9fcf108cabef None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:29.079039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-622770ed-63ff-43bc-a98f-9fcf108cabef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:29.284245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622770ed-63ff-43bc-a98f-9fcf108cabef None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:29.301483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:29.302860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:29.303208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:29.303573 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] PATH_INFO: `/v3/projects/a0a7556849be4e80a1e56245a04d0749` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:29.306459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a7556849be4e80a1e56245a04d0749)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:29.316314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:29.405114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a7556849be4e80a1e56245a04d0749', 'request_id': 'req-3bb13965-738d-4589-b761-e8e163d13d74'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:29.415041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a7556849be4e80a1e56245a04d0749', 'request_id': 'req-3bb13965-738d-4589-b761-e8e163d13d74'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:29.426507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a7556849be4e80a1e56245a04d0749', 'request_id': 'req-3bb13965-738d-4589-b761-e8e163d13d74'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:29.447030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] The token cache is being invalidate because project a0a7556849be4e80a1e56245a04d0749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:29.447328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:29.448217 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3bb13965-738d-4589-b761-e8e163d13d74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:29.453629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 907/1818] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:29 2022] DELETE /identity/v3/projects/a0a7556849be4e80a1e56245a04d0749 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:56:29.455586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1eaf0-0bce-4536-bac3-78500ac68010 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:29.455879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1eaf0-0bce-4536-bac3-78500ac68010 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:29.456164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1eaf0-0bce-4536-bac3-78500ac68010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:29.480759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-42f1eaf0-0bce-4536-bac3-78500ac68010 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:29.483457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-42f1eaf0-0bce-4536-bac3-78500ac68010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:29.628460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 912/1819] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:29.636354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7d335-3a96-45f1-af2a-c85e0b8f216a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:29.636720 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7d335-3a96-45f1-af2a-c85e0b8f216a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:29.637079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7d335-3a96-45f1-af2a-c85e0b8f216a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:29.678448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1cb7d335-3a96-45f1-af2a-c85e0b8f216a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:29.679488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1cb7d335-3a96-45f1-af2a-c85e0b8f216a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:29.698418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4837ed-10b4-4baf-aba7-cf1583f23c93 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:29.698418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4837ed-10b4-4baf-aba7-cf1583f23c93 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:29.698418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4837ed-10b4-4baf-aba7-cf1583f23c93 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:29.701693 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 913/1820] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 13:56:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:56:29.712215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4ab19-590f-4176-8c9f-7be8d34b748e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:29.712796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4ab19-590f-4176-8c9f-7be8d34b748e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:29.714006 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4ab19-590f-4176-8c9f-7be8d34b748e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:29.730953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 908/1821] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:29.731688 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7d335-3a96-45f1-af2a-c85e0b8f216a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:29.749418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7d335-3a96-45f1-af2a-c85e0b8f216a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:29.751500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-32c4ab19-590f-4176-8c9f-7be8d34b748e None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:29.755054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-32c4ab19-590f-4176-8c9f-7be8d34b748e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:29.783282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d893343-ba98-4081-853b-5c0681b238a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:29.784165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d893343-ba98-4081-853b-5c0681b238a7 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:29.784552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d893343-ba98-4081-853b-5c0681b238a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:29.784932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d893343-ba98-4081-853b-5c0681b238a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:29.792711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d893343-ba98-4081-853b-5c0681b238a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:56:29.792711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d893343-ba98-4081-853b-5c0681b238a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:29.810323 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d893343-ba98-4081-853b-5c0681b238a7 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:29.864262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 909/1822] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:56:29.866071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d893343-ba98-4081-853b-5c0681b238a7 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:29.937291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d893343-ba98-4081-853b-5c0681b238a7 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:29.975832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7754d55-1374-4bac-b69e-48fe5f9b14b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:29.975832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754d55-1374-4bac-b69e-48fe5f9b14b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:29.975832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754d55-1374-4bac-b69e-48fe5f9b14b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:29.975832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754d55-1374-4bac-b69e-48fe5f9b14b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:29.976277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7754d55-1374-4bac-b69e-48fe5f9b14b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:29.980690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7754d55-1374-4bac-b69e-48fe5f9b14b1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:30.004155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 914/1823] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 13:56:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:30.057380 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 910/1824] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:56:29 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:30.062474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4ab19-590f-4176-8c9f-7be8d34b748e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:30.079113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4ab19-590f-4176-8c9f-7be8d34b748e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:30.194444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f469d9-d307-487e-b23a-97054e952f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:30.195551 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f469d9-d307-487e-b23a-97054e952f8c admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:30.196202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f469d9-d307-487e-b23a-97054e952f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:30.196712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f469d9-d307-487e-b23a-97054e952f8c admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:30.197648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f469d9-d307-487e-b23a-97054e952f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:30.200400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f469d9-d307-487e-b23a-97054e952f8c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:30.245615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 915/1825] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:56:30 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:30.251634 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7754d55-1374-4bac-b69e-48fe5f9b14b1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:30.268169 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7754d55-1374-4bac-b69e-48fe5f9b14b1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:30.278440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0f902-6ebc-4cbf-b5da-2e2490250cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:30.279700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0f902-6ebc-4cbf-b5da-2e2490250cbf admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:30.280199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0f902-6ebc-4cbf-b5da-2e2490250cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:30.281503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0f902-6ebc-4cbf-b5da-2e2490250cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:30.283132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0f902-6ebc-4cbf-b5da-2e2490250cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:30.287416 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0f902-6ebc-4cbf-b5da-2e2490250cbf admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:30.317746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 911/1826] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:30.322213 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f469d9-d307-487e-b23a-97054e952f8c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:30.339306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f469d9-d307-487e-b23a-97054e952f8c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:30.345988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648ea59-6b17-407c-8404-312c42b3b429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:30.346763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648ea59-6b17-407c-8404-312c42b3b429 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:30.347123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648ea59-6b17-407c-8404-312c42b3b429 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:30.347554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648ea59-6b17-407c-8404-312c42b3b429 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:30.348313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648ea59-6b17-407c-8404-312c42b3b429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:30.350098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648ea59-6b17-407c-8404-312c42b3b429 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:30.365915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 916/1827] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:30.372213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0f902-6ebc-4cbf-b5da-2e2490250cbf admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:30.393738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0f902-6ebc-4cbf-b5da-2e2490250cbf admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:30.425004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28d019-7974-4fb8-aff4-bdd4507b4e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:30.425004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28d019-7974-4fb8-aff4-bdd4507b4e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:30.425004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28d019-7974-4fb8-aff4-bdd4507b4e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:30.425004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28d019-7974-4fb8-aff4-bdd4507b4e2a admin admin] PATH_INFO: `/v3/projects/085bc0056ec64b1c8af19d5c10b9f83d/users/5709b872ee754b38877df00dd14a6f97/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:30.471981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28d019-7974-4fb8-aff4-bdd4507b4e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=085bc0056ec64b1c8af19d5c10b9f83d, user_id=5709b872ee754b38877df00dd14a6f97, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:30.477407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28d019-7974-4fb8-aff4-bdd4507b4e2a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:30.511182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 912/1828] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:30 2022] PUT /identity/v3/projects/085bc0056ec64b1c8af19d5c10b9f83d/users/5709b872ee754b38877df00dd14a6f97/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:30.523380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3758e4f-4c9b-4c57-b2d7-b272f7e0d3ec None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:30.523792 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3758e4f-4c9b-4c57-b2d7-b272f7e0d3ec None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:30.523946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3758e4f-4c9b-4c57-b2d7-b272f7e0d3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:30.564200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b3758e4f-4c9b-4c57-b2d7-b272f7e0d3ec None None] MFA Rules not processed for user `5709b872ee754b38877df00dd14a6f97`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:30.565738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b3758e4f-4c9b-4c57-b2d7-b272f7e0d3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:30.773724 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 917/1829] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:30.789240 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28d019-7974-4fb8-aff4-bdd4507b4e2a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:30.793280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-cc28d019-7974-4fb8-aff4-bdd4507b4e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:30.970745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28d019-7974-4fb8-aff4-bdd4507b4e2a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:30.989710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f27b3-88f3-41c0-850c-e0db3b11e4b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:30.991965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f27b3-88f3-41c0-850c-e0db3b11e4b8 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:30.994057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f27b3-88f3-41c0-850c-e0db3b11e4b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:31.002555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f27b3-88f3-41c0-850c-e0db3b11e4b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:31.032836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f27b3-88f3-41c0-850c-e0db3b11e4b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:31.036730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f27b3-88f3-41c0-850c-e0db3b11e4b8 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:31.062483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 913/1830] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:31.381941 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3758e4f-4c9b-4c57-b2d7-b272f7e0d3ec None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:31.383461 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b3758e4f-4c9b-4c57-b2d7-b272f7e0d3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:31.579545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3758e4f-4c9b-4c57-b2d7-b272f7e0d3ec None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:31.638564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181a591-e8fe-4de6-8575-ef470b8fe579 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:31.643460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181a591-e8fe-4de6-8575-ef470b8fe579 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:31.643460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181a591-e8fe-4de6-8575-ef470b8fe579 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:31.643460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181a591-e8fe-4de6-8575-ef470b8fe579 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:31.705628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181a591-e8fe-4de6-8575-ef470b8fe579 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:31.712518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181a591-e8fe-4de6-8575-ef470b8fe579 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:31.764785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 918/1831] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:33.021096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5016f2-9b54-43ae-8bbd-d24f805a4e16 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:33.022299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5016f2-9b54-43ae-8bbd-d24f805a4e16 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:33.022859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5016f2-9b54-43ae-8bbd-d24f805a4e16 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:33.025171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 914/1832] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 13:56:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:56:33.157447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022921d-281d-470a-869f-1677d067e4e6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:33.158272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022921d-281d-470a-869f-1677d067e4e6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:33.158812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022921d-281d-470a-869f-1677d067e4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:33.237360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0022921d-281d-470a-869f-1677d067e4e6 None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:33.239146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0022921d-281d-470a-869f-1677d067e4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:33.297180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 919/1833] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 13:56:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:34.094382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5016f2-9b54-43ae-8bbd-d24f805a4e16 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:34.130557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5016f2-9b54-43ae-8bbd-d24f805a4e16 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:34.146107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904564c-6274-4f01-9ac2-39bcdfae6037 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:34.147210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904564c-6274-4f01-9ac2-39bcdfae6037 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:34.147684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904564c-6274-4f01-9ac2-39bcdfae6037 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:34.148108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904564c-6274-4f01-9ac2-39bcdfae6037 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:34.169163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904564c-6274-4f01-9ac2-39bcdfae6037 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:34.172663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904564c-6274-4f01-9ac2-39bcdfae6037 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:34.196933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 915/1834] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:39.859190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022921d-281d-470a-869f-1677d067e4e6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:39.941392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022921d-281d-470a-869f-1677d067e4e6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:39.949703 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:39.950636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:39.950906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:39.951329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] PATH_INFO: `/v3/users/5709b872ee754b38877df00dd14a6f97` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:39.952655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5709b872ee754b38877df00dd14a6f97)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:39.955284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:40.016577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5709b872ee754b38877df00dd14a6f97'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:40.032224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5709b872ee754b38877df00dd14a6f97'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:40.040327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5709b872ee754b38877df00dd14a6f97'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:40.041170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5709b872ee754b38877df00dd14a6f97'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:40.051563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5709b872ee754b38877df00dd14a6f97'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:40.064071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 920/1835] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:56:39 2022] DELETE /identity/v3/users/5709b872ee754b38877df00dd14a6f97 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:40.066471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57afd10-07a0-4500-8139-0a2184a66c89 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:40.067344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57afd10-07a0-4500-8139-0a2184a66c89 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:40.067734 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57afd10-07a0-4500-8139-0a2184a66c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:40.091508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b57afd10-07a0-4500-8139-0a2184a66c89 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:40.093416 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b57afd10-07a0-4500-8139-0a2184a66c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:40.250820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:40.253408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:40.259224 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 916/1836] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:40.354016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c489ce-91bf-4af3-8e73-882a7e13398d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:40.357978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee3115-0f26-48a1-ad57-f2d7dcf40837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:40.359353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee3115-0f26-48a1-ad57-f2d7dcf40837 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:40.359998 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee3115-0f26-48a1-ad57-f2d7dcf40837 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:40.360273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee3115-0f26-48a1-ad57-f2d7dcf40837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:40.370082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee3115-0f26-48a1-ad57-f2d7dcf40837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:40.371586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee3115-0f26-48a1-ad57-f2d7dcf40837 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:40.381734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 921/1837] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:40.756857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57afd10-07a0-4500-8139-0a2184a66c89 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:40.758511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b57afd10-07a0-4500-8139-0a2184a66c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:40.771291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57afd10-07a0-4500-8139-0a2184a66c89 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:40.776079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:40.776959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:40.777237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:40.777513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] PATH_INFO: `/v3/projects/085bc0056ec64b1c8af19d5c10b9f83d` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:40.778951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] RBAC: Authorizing `identity:delete_project(project_id=085bc0056ec64b1c8af19d5c10b9f83d)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:40.781375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:40.856700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '085bc0056ec64b1c8af19d5c10b9f83d', 'request_id': 'req-1ed48d5e-e4c2-4581-9ded-ef161e20526d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:40.862963 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '085bc0056ec64b1c8af19d5c10b9f83d', 'request_id': 'req-1ed48d5e-e4c2-4581-9ded-ef161e20526d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:40.868998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '085bc0056ec64b1c8af19d5c10b9f83d', 'request_id': 'req-1ed48d5e-e4c2-4581-9ded-ef161e20526d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:40.883345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] The token cache is being invalidate because project 085bc0056ec64b1c8af19d5c10b9f83d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:40.883637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:40.884298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ed48d5e-e4c2-4581-9ded-ef161e20526d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:40.885937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 917/1838] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:40 2022] DELETE /identity/v3/projects/085bc0056ec64b1c8af19d5c10b9f83d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:40.891779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c368c-4cd8-46d0-9577-ec43cfe5b0e6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:40.892259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c368c-4cd8-46d0-9577-ec43cfe5b0e6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:40.892358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c368c-4cd8-46d0-9577-ec43cfe5b0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:40.911143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-ff1c368c-4cd8-46d0-9577-ec43cfe5b0e6 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:40.912392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ff1c368c-4cd8-46d0-9577-ec43cfe5b0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:40.978354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 922/1839] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:40.986275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7dd39-0e04-4a77-a840-c18084633e67 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:40.986555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7dd39-0e04-4a77-a840-c18084633e67 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:40.986866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7dd39-0e04-4a77-a840-c18084633e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:41.005345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-41c7dd39-0e04-4a77-a840-c18084633e67 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:41.006554 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-41c7dd39-0e04-4a77-a840-c18084633e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:41.014679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 918/1840] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:41.019670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c368c-4cd8-46d0-9577-ec43cfe5b0e6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:41.029761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c368c-4cd8-46d0-9577-ec43cfe5b0e6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:41.034791 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821994f6-2ad2-4301-a376-874e997a015d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:41.035465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821994f6-2ad2-4301-a376-874e997a015d admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:41.035746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821994f6-2ad2-4301-a376-874e997a015d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:41.036020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821994f6-2ad2-4301-a376-874e997a015d admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:41.036896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821994f6-2ad2-4301-a376-874e997a015d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:56:41.037508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821994f6-2ad2-4301-a376-874e997a015d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:41.039147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821994f6-2ad2-4301-a376-874e997a015d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:41.051255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 923/1841] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:56:41.055574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7dd39-0e04-4a77-a840-c18084633e67 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:41.065655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7dd39-0e04-4a77-a840-c18084633e67 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:41.070800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c570d-90a2-4854-9f17-86855ff2e7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:41.071530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c570d-90a2-4854-9f17-86855ff2e7eb admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:41.071760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c570d-90a2-4854-9f17-86855ff2e7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:41.072032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c570d-90a2-4854-9f17-86855ff2e7eb admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:41.072721 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c570d-90a2-4854-9f17-86855ff2e7eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:41.074689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c570d-90a2-4854-9f17-86855ff2e7eb admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:41.111943 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 919/1842] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:56:41 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:41.114407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821994f6-2ad2-4301-a376-874e997a015d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:41.126346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821994f6-2ad2-4301-a376-874e997a015d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:41.213113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85648233-393d-4112-8780-880332ee5885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:41.213800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85648233-393d-4112-8780-880332ee5885 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:41.214103 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85648233-393d-4112-8780-880332ee5885 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:41.214344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85648233-393d-4112-8780-880332ee5885 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:41.215181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85648233-393d-4112-8780-880332ee5885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:41.217156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85648233-393d-4112-8780-880332ee5885 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:41.243561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 924/1843] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:56:41 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:56:41.248335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c570d-90a2-4854-9f17-86855ff2e7eb admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:41.261152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c570d-90a2-4854-9f17-86855ff2e7eb admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:41.267019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29abbaf-0c2d-4750-b3ad-31bb34b37d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:41.267715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29abbaf-0c2d-4750-b3ad-31bb34b37d64 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:41.267992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29abbaf-0c2d-4750-b3ad-31bb34b37d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:41.268263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29abbaf-0c2d-4750-b3ad-31bb34b37d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:41.269227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29abbaf-0c2d-4750-b3ad-31bb34b37d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:41.270662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29abbaf-0c2d-4750-b3ad-31bb34b37d64 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:41.285018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 920/1844] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:56:41.289491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85648233-393d-4112-8780-880332ee5885 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:41.302991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85648233-393d-4112-8780-880332ee5885 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:41.310312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c819f-24cc-4d4b-be82-d2098a196b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:41.311084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c819f-24cc-4d4b-be82-d2098a196b3f admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:41.311373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c819f-24cc-4d4b-be82-d2098a196b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:41.312222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c819f-24cc-4d4b-be82-d2098a196b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:41.312960 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c819f-24cc-4d4b-be82-d2098a196b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:41.314665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c819f-24cc-4d4b-be82-d2098a196b3f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:41.329857 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 925/1845] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:56:41.334436 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29abbaf-0c2d-4750-b3ad-31bb34b37d64 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:41.344916 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29abbaf-0c2d-4750-b3ad-31bb34b37d64 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:41.353633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb8572-2236-460b-ba93-5686e876a11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:41.354515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8572-2236-460b-ba93-5686e876a11b admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:41.354820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8572-2236-460b-ba93-5686e876a11b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:41.355096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb8572-2236-460b-ba93-5686e876a11b admin admin] PATH_INFO: `/v3/projects/d0c77ac1d0834bd4a705cf141a24f24d/users/38ce69da3861420f9f5a0a018d05da55/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:41.372590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb8572-2236-460b-ba93-5686e876a11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c77ac1d0834bd4a705cf141a24f24d, user_id=38ce69da3861420f9f5a0a018d05da55, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:41.375880 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb8572-2236-460b-ba93-5686e876a11b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:41.390993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 921/1846] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:41 2022] PUT /identity/v3/projects/d0c77ac1d0834bd4a705cf141a24f24d/users/38ce69da3861420f9f5a0a018d05da55/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:41.402085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62ac2f-05da-4bb1-81f0-db8186b69617 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:41.402365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62ac2f-05da-4bb1-81f0-db8186b69617 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:41.402639 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62ac2f-05da-4bb1-81f0-db8186b69617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:41.431307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-8b62ac2f-05da-4bb1-81f0-db8186b69617 None None] MFA Rules not processed for user `38ce69da3861420f9f5a0a018d05da55`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:41.432753 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-8b62ac2f-05da-4bb1-81f0-db8186b69617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:41.501739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 926/1847] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:41.512624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb8572-2236-460b-ba93-5686e876a11b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:41.514843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5dbb8572-2236-460b-ba93-5686e876a11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:41.584880 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb8572-2236-460b-ba93-5686e876a11b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:41.588725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72562e24-1991-425e-bbcd-0d130007999f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:41.589381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72562e24-1991-425e-bbcd-0d130007999f service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:41.589663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72562e24-1991-425e-bbcd-0d130007999f service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:41.589939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72562e24-1991-425e-bbcd-0d130007999f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:41.598814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72562e24-1991-425e-bbcd-0d130007999f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:41.600356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72562e24-1991-425e-bbcd-0d130007999f service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:41.610787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 922/1848] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:46.746538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62ac2f-05da-4bb1-81f0-db8186b69617 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:46.899529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62ac2f-05da-4bb1-81f0-db8186b69617 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:46.916977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:46.917879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:46.918160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:46.918442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] PATH_INFO: `/v3/users/38ce69da3861420f9f5a0a018d05da55` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:46.923735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=38ce69da3861420f9f5a0a018d05da55)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:46.931224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:47.070981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38ce69da3861420f9f5a0a018d05da55'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:47.094992 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38ce69da3861420f9f5a0a018d05da55'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:47.102416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38ce69da3861420f9f5a0a018d05da55'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:47.115233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38ce69da3861420f9f5a0a018d05da55'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:47.126455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38ce69da3861420f9f5a0a018d05da55'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:47.141291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 927/1849] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:56:46 2022] DELETE /identity/v3/users/38ce69da3861420f9f5a0a018d05da55 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:47.145798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac9f69-d95c-42b9-a99d-7a0da15aff0a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:47.146320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac9f69-d95c-42b9-a99d-7a0da15aff0a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:47.151093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac9f69-d95c-42b9-a99d-7a0da15aff0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:47.230894 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e1ac9f69-d95c-42b9-a99d-7a0da15aff0a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:47.237202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e1ac9f69-d95c-42b9-a99d-7a0da15aff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:47.366803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 923/1850] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:47.416055 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:47.422845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:47.534749 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0f765-e82a-4215-9863-b0aab39df06a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:47.540512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97b257-67cf-4556-a3cb-ffe7ff46c8f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:47.541268 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97b257-67cf-4556-a3cb-ffe7ff46c8f1 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:47.541553 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97b257-67cf-4556-a3cb-ffe7ff46c8f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:47.542009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97b257-67cf-4556-a3cb-ffe7ff46c8f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:47.556118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97b257-67cf-4556-a3cb-ffe7ff46c8f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:47.558007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97b257-67cf-4556-a3cb-ffe7ff46c8f1 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:47.575436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 928/1851] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:56:48.214300 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac9f69-d95c-42b9-a99d-7a0da15aff0a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:48.217381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e1ac9f69-d95c-42b9-a99d-7a0da15aff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:48.235583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac9f69-d95c-42b9-a99d-7a0da15aff0a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:48.241313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:48.242190 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:48.242470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:48.242768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] PATH_INFO: `/v3/projects/d0c77ac1d0834bd4a705cf141a24f24d` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:48.243990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c77ac1d0834bd4a705cf141a24f24d)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:48.247093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:48.301499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c77ac1d0834bd4a705cf141a24f24d', 'request_id': 'req-fb9db601-7999-4473-a7b9-6252c31e0b92'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:48.318729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c77ac1d0834bd4a705cf141a24f24d', 'request_id': 'req-fb9db601-7999-4473-a7b9-6252c31e0b92'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:48.325668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c77ac1d0834bd4a705cf141a24f24d', 'request_id': 'req-fb9db601-7999-4473-a7b9-6252c31e0b92'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:48.343601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] The token cache is being invalidate because project d0c77ac1d0834bd4a705cf141a24f24d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:56:48.344524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:48.345363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb9db601-7999-4473-a7b9-6252c31e0b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:56:48.347763 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 924/1852] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:56:48 2022] DELETE /identity/v3/projects/d0c77ac1d0834bd4a705cf141a24f24d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:48.353335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acb2b0-16ce-4349-ab67-5e2369c53b8f None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:48.353998 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acb2b0-16ce-4349-ab67-5e2369c53b8f None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:48.353998 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acb2b0-16ce-4349-ab67-5e2369c53b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:48.377401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-79acb2b0-16ce-4349-ab67-5e2369c53b8f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:48.379265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-79acb2b0-16ce-4349-ab67-5e2369c53b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:48.504289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 929/1853] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:56:48.512066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a6c56-c4d8-46f1-b985-dcdbfccac709 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:48.512351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a6c56-c4d8-46f1-b985-dcdbfccac709 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:48.513069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a6c56-c4d8-46f1-b985-dcdbfccac709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:48.543358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f80a6c56-c4d8-46f1-b985-dcdbfccac709 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:48.544606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f80a6c56-c4d8-46f1-b985-dcdbfccac709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:48.554318 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 925/1854] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:48.559796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79acb2b0-16ce-4349-ab67-5e2369c53b8f None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:48.571350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79acb2b0-16ce-4349-ab67-5e2369c53b8f None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:48.577856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb11923-ac14-4441-8963-d3ab9e4b01cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:48.578797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb11923-ac14-4441-8963-d3ab9e4b01cf admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:48.579154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb11923-ac14-4441-8963-d3ab9e4b01cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:48.579499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb11923-ac14-4441-8963-d3ab9e4b01cf admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:48.580780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb11923-ac14-4441-8963-d3ab9e4b01cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:56:48.581166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb11923-ac14-4441-8963-d3ab9e4b01cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:48.582971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb11923-ac14-4441-8963-d3ab9e4b01cf admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:48.617299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 930/1855] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:56:48.620748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a6c56-c4d8-46f1-b985-dcdbfccac709 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:48.633408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a6c56-c4d8-46f1-b985-dcdbfccac709 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:48.644020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db87d9-cace-418e-9def-19675942bf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:48.644769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db87d9-cace-418e-9def-19675942bf3b admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:48.645056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db87d9-cace-418e-9def-19675942bf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:48.645338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db87d9-cace-418e-9def-19675942bf3b admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:48.648130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db87d9-cace-418e-9def-19675942bf3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:48.651552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db87d9-cace-418e-9def-19675942bf3b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:48.733979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 926/1856] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:56:48 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:56:48.736466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb11923-ac14-4441-8963-d3ab9e4b01cf admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:48.755742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb11923-ac14-4441-8963-d3ab9e4b01cf admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:48.888164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5aaf6-905b-4c2d-8385-c985709e1850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:48.900263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5aaf6-905b-4c2d-8385-c985709e1850 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:48.901041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5aaf6-905b-4c2d-8385-c985709e1850 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:48.901738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5aaf6-905b-4c2d-8385-c985709e1850 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:48.902694 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5aaf6-905b-4c2d-8385-c985709e1850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:48.905674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5aaf6-905b-4c2d-8385-c985709e1850 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:48.942111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 931/1857] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:56:48 2022] POST /identity/v3/users => generated 284 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:56:48.946095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db87d9-cace-418e-9def-19675942bf3b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:48.965177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db87d9-cace-418e-9def-19675942bf3b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:48.972024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a14c3-33b3-40ab-8f7b-c5e9146a5e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:48.973006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a14c3-33b3-40ab-8f7b-c5e9146a5e1e admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:48.973388 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a14c3-33b3-40ab-8f7b-c5e9146a5e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:48.973795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a14c3-33b3-40ab-8f7b-c5e9146a5e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:48.974909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a14c3-33b3-40ab-8f7b-c5e9146a5e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:48.977010 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a14c3-33b3-40ab-8f7b-c5e9146a5e1e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:48.991798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 927/1858] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:56:48.995590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5aaf6-905b-4c2d-8385-c985709e1850 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:49.021773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5aaf6-905b-4c2d-8385-c985709e1850 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:49.029184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf11e5-1914-4ab2-8756-43872cf66e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:49.030882 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf11e5-1914-4ab2-8756-43872cf66e5d admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:49.031730 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf11e5-1914-4ab2-8756-43872cf66e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:49.031964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf11e5-1914-4ab2-8756-43872cf66e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:49.033340 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf11e5-1914-4ab2-8756-43872cf66e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:49.037878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf11e5-1914-4ab2-8756-43872cf66e5d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:49.058464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 932/1859] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:56:49.061907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a14c3-33b3-40ab-8f7b-c5e9146a5e1e admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:49.076087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a14c3-33b3-40ab-8f7b-c5e9146a5e1e admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:49.083367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a40b6-fb37-42f4-a56e-beac2a6da012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:49.084803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a40b6-fb37-42f4-a56e-beac2a6da012 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:49.085310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a40b6-fb37-42f4-a56e-beac2a6da012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:49.085794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a40b6-fb37-42f4-a56e-beac2a6da012 admin admin] PATH_INFO: `/v3/projects/eef6814e331548b092b90ddcf3665b66/users/ee7aeeef29a2456ca21d74d27ec08167/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:49.105968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a40b6-fb37-42f4-a56e-beac2a6da012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef6814e331548b092b90ddcf3665b66, user_id=ee7aeeef29a2456ca21d74d27ec08167, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:49.110112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a40b6-fb37-42f4-a56e-beac2a6da012 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:49.135449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 928/1860] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:56:49 2022] PUT /identity/v3/projects/eef6814e331548b092b90ddcf3665b66/users/ee7aeeef29a2456ca21d74d27ec08167/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:56:49.144289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67a69d-9cca-4577-85fe-43db7a3e257c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:49.144571 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67a69d-9cca-4577-85fe-43db7a3e257c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:49.144994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67a69d-9cca-4577-85fe-43db7a3e257c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:49.177459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-de67a69d-9cca-4577-85fe-43db7a3e257c None None] MFA Rules not processed for user `ee7aeeef29a2456ca21d74d27ec08167`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:56:49.178744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-de67a69d-9cca-4577-85fe-43db7a3e257c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:49.257352 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 933/1861] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:56:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:56:49.276546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a40b6-fb37-42f4-a56e-beac2a6da012 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:56:49.279903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-329a40b6-fb37-42f4-a56e-beac2a6da012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:56:49.370139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a40b6-fb37-42f4-a56e-beac2a6da012 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:56:49.374657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda03c56-a358-4a39-b812-a118f1c143ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4f45e859d84ceaa5a01477e83e098e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:56:49.375409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda03c56-a358-4a39-b812-a118f1c143ac service glance] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:56:49.375697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda03c56-a358-4a39-b812-a118f1c143ac service glance] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:56:49.376018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda03c56-a358-4a39-b812-a118f1c143ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:56:49.384989 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda03c56-a358-4a39-b812-a118f1c143ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:56:49.386420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda03c56-a358-4a39-b812-a118f1c143ac service glance] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:56:49.396689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 929/1862] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:56:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:00.672697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67a69d-9cca-4577-85fe-43db7a3e257c None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:00.674367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-de67a69d-9cca-4577-85fe-43db7a3e257c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:00.759799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67a69d-9cca-4577-85fe-43db7a3e257c None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:00.764744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:00.766328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:00.766328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:00.766522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] PATH_INFO: `/v3/users/b0cbbded7575465f9576def2948e12de` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:00.767889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0cbbded7575465f9576def2948e12de)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:00.769925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:00.823457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0cbbded7575465f9576def2948e12de'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:00.837883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0cbbded7575465f9576def2948e12de'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:00.844854 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0cbbded7575465f9576def2948e12de'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:00.845593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0cbbded7575465f9576def2948e12de'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:00.853309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0cbbded7575465f9576def2948e12de'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:00.860679 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 934/1863] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:00 2022] DELETE /identity/v3/users/b0cbbded7575465f9576def2948e12de => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:00.865384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84150a-2361-4abd-9a65-b54b87cd415f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:00.866313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84150a-2361-4abd-9a65-b54b87cd415f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:00.866699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84150a-2361-4abd-9a65-b54b87cd415f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:00.886040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-5a84150a-2361-4abd-9a65-b54b87cd415f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:00.887297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5a84150a-2361-4abd-9a65-b54b87cd415f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:00.952456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 930/1864] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:01.006258 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:01.010402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:01.090399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd1168-40b6-4947-8fd1-1a5df319e532 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:01.095047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222818d7-b063-4d61-abdf-042d2ed71127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:01.095723 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222818d7-b063-4d61-abdf-042d2ed71127 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:01.096005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222818d7-b063-4d61-abdf-042d2ed71127 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:01.096280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222818d7-b063-4d61-abdf-042d2ed71127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:01.106302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222818d7-b063-4d61-abdf-042d2ed71127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:01.108388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222818d7-b063-4d61-abdf-042d2ed71127 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:01.128309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 935/1865] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:02.570480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84150a-2361-4abd-9a65-b54b87cd415f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:02.572565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5a84150a-2361-4abd-9a65-b54b87cd415f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:02.594950 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84150a-2361-4abd-9a65-b54b87cd415f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:02.601188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:02.602650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:02.603206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:02.603925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] PATH_INFO: `/v3/projects/31a154cba8db4ed58594e81e7028648e` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:02.605620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a154cba8db4ed58594e81e7028648e)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:02.609261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:02.692868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a154cba8db4ed58594e81e7028648e', 'request_id': 'req-7ffda351-abb4-475a-817e-ed023cde092f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:02.709349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a154cba8db4ed58594e81e7028648e', 'request_id': 'req-7ffda351-abb4-475a-817e-ed023cde092f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:02.717061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a154cba8db4ed58594e81e7028648e', 'request_id': 'req-7ffda351-abb4-475a-817e-ed023cde092f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:02.754282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] The token cache is being invalidate because project 31a154cba8db4ed58594e81e7028648e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:02.755052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:02.755939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:02.758092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 931/1866] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:02 2022] DELETE /identity/v3/projects/31a154cba8db4ed58594e81e7028648e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:02.762675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222818d7-b063-4d61-abdf-042d2ed71127 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:02.775761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222818d7-b063-4d61-abdf-042d2ed71127 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:02.872874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:02.874829 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:02.875612 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:02.876643 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] PATH_INFO: `/v3/users/053d8ab14ec1455a8672142eeb1d1a4f` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:02.878290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=053d8ab14ec1455a8672142eeb1d1a4f)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:02.880561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:02.949213 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '053d8ab14ec1455a8672142eeb1d1a4f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:02.980584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '053d8ab14ec1455a8672142eeb1d1a4f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:02.990242 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '053d8ab14ec1455a8672142eeb1d1a4f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:02.991280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '053d8ab14ec1455a8672142eeb1d1a4f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:03.007879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d7e2a3-38aa-44a0-a186-7aa610193173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '053d8ab14ec1455a8672142eeb1d1a4f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:03.016972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 936/1867] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:02 2022] DELETE /identity/v3/users/053d8ab14ec1455a8672142eeb1d1a4f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:03.653533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:03.655181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:03.672038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffda351-abb4-475a-817e-ed023cde092f admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:03.772584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:03.773244 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:03.773524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:03.773800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] PATH_INFO: `/v3/projects/4e84895304ef46cd92ff838c34ccafa7` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:03.775356 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e84895304ef46cd92ff838c34ccafa7)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:03.777359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:03.841435 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e84895304ef46cd92ff838c34ccafa7', 'request_id': 'req-157e81e1-1e4a-413b-a5ee-a546e65bf427'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:03.851153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e84895304ef46cd92ff838c34ccafa7', 'request_id': 'req-157e81e1-1e4a-413b-a5ee-a546e65bf427'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:03.858824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e84895304ef46cd92ff838c34ccafa7', 'request_id': 'req-157e81e1-1e4a-413b-a5ee-a546e65bf427'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:03.874132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] The token cache is being invalidate because project 4e84895304ef46cd92ff838c34ccafa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:03.874424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:03.880443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-157e81e1-1e4a-413b-a5ee-a546e65bf427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:03.882285 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 932/1868] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:03 2022] DELETE /identity/v3/projects/4e84895304ef46cd92ff838c34ccafa7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:03.887501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae60271-5090-4fe0-932a-3b8fbaaf97eb None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:03.887994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae60271-5090-4fe0-932a-3b8fbaaf97eb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:03.888222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae60271-5090-4fe0-932a-3b8fbaaf97eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:03.909411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-aae60271-5090-4fe0-932a-3b8fbaaf97eb None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:03.910654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-aae60271-5090-4fe0-932a-3b8fbaaf97eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:03.990997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 937/1869] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:03.999606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d4595-4450-44b8-b3ed-d687905bb7b7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:04.000122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d4595-4450-44b8-b3ed-d687905bb7b7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:04.000555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d4595-4450-44b8-b3ed-d687905bb7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:04.020530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c85d4595-4450-44b8-b3ed-d687905bb7b7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:04.021738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c85d4595-4450-44b8-b3ed-d687905bb7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:04.030368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 933/1870] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:04.034663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae60271-5090-4fe0-932a-3b8fbaaf97eb None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:04.047368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae60271-5090-4fe0-932a-3b8fbaaf97eb None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:04.063584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68360e-b312-4e75-a0f6-b71817ebc571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:04.064186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68360e-b312-4e75-a0f6-b71817ebc571 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:04.064458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68360e-b312-4e75-a0f6-b71817ebc571 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:04.064740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68360e-b312-4e75-a0f6-b71817ebc571 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:04.065650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68360e-b312-4e75-a0f6-b71817ebc571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:04.065952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68360e-b312-4e75-a0f6-b71817ebc571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:04.067842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68360e-b312-4e75-a0f6-b71817ebc571 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:04.084312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 938/1871] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:57:04.088754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d4595-4450-44b8-b3ed-d687905bb7b7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:04.104663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d4595-4450-44b8-b3ed-d687905bb7b7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:04.112726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08be1b-783b-45fb-aec8-dd8b4906fe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:04.113754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08be1b-783b-45fb-aec8-dd8b4906fe60 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:04.114230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08be1b-783b-45fb-aec8-dd8b4906fe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:04.114595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08be1b-783b-45fb-aec8-dd8b4906fe60 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:04.115483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08be1b-783b-45fb-aec8-dd8b4906fe60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:04.117519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08be1b-783b-45fb-aec8-dd8b4906fe60 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:04.156495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 934/1872] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:04 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:04.160665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68360e-b312-4e75-a0f6-b71817ebc571 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:04.174472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68360e-b312-4e75-a0f6-b71817ebc571 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:04.245707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54dba9-2c65-49c0-a0e0-904991ee6e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:04.246505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54dba9-2c65-49c0-a0e0-904991ee6e00 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:04.246883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54dba9-2c65-49c0-a0e0-904991ee6e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:04.247230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54dba9-2c65-49c0-a0e0-904991ee6e00 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:04.248585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54dba9-2c65-49c0-a0e0-904991ee6e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:04.250656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54dba9-2c65-49c0-a0e0-904991ee6e00 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:04.274845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 939/1873] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:04 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:04.279636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08be1b-783b-45fb-aec8-dd8b4906fe60 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:04.294862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08be1b-783b-45fb-aec8-dd8b4906fe60 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:04.303425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f90ca5-d73f-4bdc-add0-d63240c5e511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:04.304600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f90ca5-d73f-4bdc-add0-d63240c5e511 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:04.305111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f90ca5-d73f-4bdc-add0-d63240c5e511 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:04.305623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f90ca5-d73f-4bdc-add0-d63240c5e511 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:04.307344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f90ca5-d73f-4bdc-add0-d63240c5e511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:04.310750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f90ca5-d73f-4bdc-add0-d63240c5e511 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:04.330962 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 935/1874] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:04.335204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54dba9-2c65-49c0-a0e0-904991ee6e00 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:04.354138 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54dba9-2c65-49c0-a0e0-904991ee6e00 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:04.364244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a6d14-e1ed-4ea5-b4b3-7699228207a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:04.365059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a6d14-e1ed-4ea5-b4b3-7699228207a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:04.365455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a6d14-e1ed-4ea5-b4b3-7699228207a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:04.365845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a6d14-e1ed-4ea5-b4b3-7699228207a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:04.366738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a6d14-e1ed-4ea5-b4b3-7699228207a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:04.368689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a6d14-e1ed-4ea5-b4b3-7699228207a9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:04.391106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 940/1875] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:57:04.396534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f90ca5-d73f-4bdc-add0-d63240c5e511 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:04.417555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f90ca5-d73f-4bdc-add0-d63240c5e511 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:04.426051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ac575-f54e-40f9-813e-5a973c08d9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:04.427003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ac575-f54e-40f9-813e-5a973c08d9b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:04.427288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ac575-f54e-40f9-813e-5a973c08d9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:04.427570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ac575-f54e-40f9-813e-5a973c08d9b0 admin admin] PATH_INFO: `/v3/projects/be8784249e65481f8b36e5a90f7b4fa6/users/0923eee9fae1420995ef847d1a3b9aea/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:04.448484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ac575-f54e-40f9-813e-5a973c08d9b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be8784249e65481f8b36e5a90f7b4fa6, user_id=0923eee9fae1420995ef847d1a3b9aea, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:04.451778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ac575-f54e-40f9-813e-5a973c08d9b0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:04.485670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 936/1876] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:04 2022] PUT /identity/v3/projects/be8784249e65481f8b36e5a90f7b4fa6/users/0923eee9fae1420995ef847d1a3b9aea/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:04.491889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a5e99-3ee2-4de3-9c56-17ea3c5752fb None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:04.492175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a5e99-3ee2-4de3-9c56-17ea3c5752fb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:04.492475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a5e99-3ee2-4de3-9c56-17ea3c5752fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:04.529067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f01a5e99-3ee2-4de3-9c56-17ea3c5752fb None None] MFA Rules not processed for user `0923eee9fae1420995ef847d1a3b9aea`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:04.531294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f01a5e99-3ee2-4de3-9c56-17ea3c5752fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:04.633443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 941/1877] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:04.640290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ac575-f54e-40f9-813e-5a973c08d9b0 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:04.734588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ac575-f54e-40f9-813e-5a973c08d9b0 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:04.760258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d3b0-7180-4531-ac22-71b5423b6809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:04.761502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d3b0-7180-4531-ac22-71b5423b6809 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:04.761502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d3b0-7180-4531-ac22-71b5423b6809 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:04.761502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d3b0-7180-4531-ac22-71b5423b6809 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:04.762195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644d3b0-7180-4531-ac22-71b5423b6809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:04.765136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644d3b0-7180-4531-ac22-71b5423b6809 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:04.806952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 937/1878] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:04 2022] POST /identity/v3/projects => generated 363 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:04.812182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a5e99-3ee2-4de3-9c56-17ea3c5752fb None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:04.839113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a5e99-3ee2-4de3-9c56-17ea3c5752fb None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:04.935596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba9627-daa1-4703-b59c-7a28594e980f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:04.937075 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba9627-daa1-4703-b59c-7a28594e980f admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:04.937467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba9627-daa1-4703-b59c-7a28594e980f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:04.937917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba9627-daa1-4703-b59c-7a28594e980f admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:04.938848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba9627-daa1-4703-b59c-7a28594e980f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:04.940999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba9627-daa1-4703-b59c-7a28594e980f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:04.991516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 942/1879] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:04 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:04.992093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d3b0-7180-4531-ac22-71b5423b6809 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:05.016618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d3b0-7180-4531-ac22-71b5423b6809 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:05.025396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46198940-e464-4b18-9e7b-c1648f50b5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:05.026093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46198940-e464-4b18-9e7b-c1648f50b5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:05.026381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46198940-e464-4b18-9e7b-c1648f50b5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:05.026663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46198940-e464-4b18-9e7b-c1648f50b5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:05.027449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46198940-e464-4b18-9e7b-c1648f50b5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:05.029642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46198940-e464-4b18-9e7b-c1648f50b5b9 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:05.045977 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 938/1880] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:57:05.051368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba9627-daa1-4703-b59c-7a28594e980f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:05.063022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba9627-daa1-4703-b59c-7a28594e980f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:05.068460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70415320-796f-44e9-a944-4935a7290ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:05.069480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70415320-796f-44e9-a944-4935a7290ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:05.069832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70415320-796f-44e9-a944-4935a7290ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:05.070177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70415320-796f-44e9-a944-4935a7290ed9 admin admin] PATH_INFO: `/v3/projects/9d64fe71849f45afa1742c32d20689a8/users/5ffc17c902bf481d892a62b29744fd37/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:05.093133 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70415320-796f-44e9-a944-4935a7290ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d64fe71849f45afa1742c32d20689a8, user_id=5ffc17c902bf481d892a62b29744fd37, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:05.097780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70415320-796f-44e9-a944-4935a7290ed9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:05.120102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 943/1881] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:05 2022] PUT /identity/v3/projects/9d64fe71849f45afa1742c32d20689a8/users/5ffc17c902bf481d892a62b29744fd37/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:05.126012 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46198940-e464-4b18-9e7b-c1648f50b5b9 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:05.136914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46198940-e464-4b18-9e7b-c1648f50b5b9 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:05.244720 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135beb47-11aa-4c1b-b997-5c1ee5f5d241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:05.245448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135beb47-11aa-4c1b-b997-5c1ee5f5d241 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:05.245736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135beb47-11aa-4c1b-b997-5c1ee5f5d241 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:05.246008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135beb47-11aa-4c1b-b997-5c1ee5f5d241 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:05.246780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135beb47-11aa-4c1b-b997-5c1ee5f5d241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:05.248541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135beb47-11aa-4c1b-b997-5c1ee5f5d241 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:05.266865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 939/1882] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:05.269650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70415320-796f-44e9-a944-4935a7290ed9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:05.286479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70415320-796f-44e9-a944-4935a7290ed9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:05.295734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa8bc1-55d0-4a3b-859f-216ee33a8903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:05.296921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa8bc1-55d0-4a3b-859f-216ee33a8903 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:05.298901 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa8bc1-55d0-4a3b-859f-216ee33a8903 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:05.298901 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa8bc1-55d0-4a3b-859f-216ee33a8903 admin admin] PATH_INFO: `/v3/projects/9d64fe71849f45afa1742c32d20689a8/users/5ffc17c902bf481d892a62b29744fd37/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:05.300510 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa8bc1-55d0-4a3b-859f-216ee33a8903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d64fe71849f45afa1742c32d20689a8, user_id=5ffc17c902bf481d892a62b29744fd37, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:05.303451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa8bc1-55d0-4a3b-859f-216ee33a8903 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:05.324799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 944/1883] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:05 2022] PUT /identity/v3/projects/9d64fe71849f45afa1742c32d20689a8/users/5ffc17c902bf481d892a62b29744fd37/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:05.336619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb5442-4e81-428f-a68f-9fea8b6bfd8c None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:05.337270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb5442-4e81-428f-a68f-9fea8b6bfd8c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:05.337728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb5442-4e81-428f-a68f-9fea8b6bfd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:05.379484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-bebb5442-4e81-428f-a68f-9fea8b6bfd8c None None] MFA Rules not processed for user `5ffc17c902bf481d892a62b29744fd37`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:05.381189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-bebb5442-4e81-428f-a68f-9fea8b6bfd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:05.507870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 940/1884] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:05 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:05.514041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa8bc1-55d0-4a3b-859f-216ee33a8903 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:05.516514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a1fa8bc1-55d0-4a3b-859f-216ee33a8903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:05.604735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa8bc1-55d0-4a3b-859f-216ee33a8903 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:05.611474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b065a-51b2-49fd-b570-0dbd3fa76e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:05.613191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b065a-51b2-49fd-b570-0dbd3fa76e69 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:05.613712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b065a-51b2-49fd-b570-0dbd3fa76e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:05.614209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b065a-51b2-49fd-b570-0dbd3fa76e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:05.630059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b065a-51b2-49fd-b570-0dbd3fa76e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:05.631570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b065a-51b2-49fd-b570-0dbd3fa76e69 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:05.643361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 945/1885] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:05 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:05.753134 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb5442-4e81-428f-a68f-9fea8b6bfd8c None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:05.764972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb5442-4e81-428f-a68f-9fea8b6bfd8c None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:05.769442 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8312ade-d788-4eb5-96a2-9a75ca9419fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:05.770101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8312ade-d788-4eb5-96a2-9a75ca9419fb service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:05.770398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8312ade-d788-4eb5-96a2-9a75ca9419fb service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:05.770656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8312ade-d788-4eb5-96a2-9a75ca9419fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:05.848577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8312ade-d788-4eb5-96a2-9a75ca9419fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:05.855964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8312ade-d788-4eb5-96a2-9a75ca9419fb service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:05.869512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 941/1886] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:21.449537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b065a-51b2-49fd-b570-0dbd3fa76e69 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:21.625750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b065a-51b2-49fd-b570-0dbd3fa76e69 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:21.631714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:21.632614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:21.632896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:21.633170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] PATH_INFO: `/v3/users/0923eee9fae1420995ef847d1a3b9aea` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:21.634728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0923eee9fae1420995ef847d1a3b9aea)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:21.636671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:21.702037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0923eee9fae1420995ef847d1a3b9aea'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:21.717717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0923eee9fae1420995ef847d1a3b9aea'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:21.726756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0923eee9fae1420995ef847d1a3b9aea'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:21.727588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0923eee9fae1420995ef847d1a3b9aea'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:21.741772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0923eee9fae1420995ef847d1a3b9aea'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:21.750236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 946/1887] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:21 2022] DELETE /identity/v3/users/0923eee9fae1420995ef847d1a3b9aea => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:21.754886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55485cd5-9630-4f7b-b691-26040d98f9ee None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:21.755497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55485cd5-9630-4f7b-b691-26040d98f9ee None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:21.755497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55485cd5-9630-4f7b-b691-26040d98f9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:21.822976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-55485cd5-9630-4f7b-b691-26040d98f9ee None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:21.822976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-55485cd5-9630-4f7b-b691-26040d98f9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:21.917336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 942/1888] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:21.947043 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:21.948845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:22.088548 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d77900-1b11-49ed-abbf-04e1874fbf10 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:22.092712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fb86f-938c-490f-8c24-2f36d0cbd6ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:22.093541 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fb86f-938c-490f-8c24-2f36d0cbd6ec service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:22.093925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fb86f-938c-490f-8c24-2f36d0cbd6ec service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:22.094310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fb86f-938c-490f-8c24-2f36d0cbd6ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:22.106222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fb86f-938c-490f-8c24-2f36d0cbd6ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:22.110969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fb86f-938c-490f-8c24-2f36d0cbd6ec service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:22.125743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 947/1889] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:22.702199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55485cd5-9630-4f7b-b691-26040d98f9ee None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:22.703980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-55485cd5-9630-4f7b-b691-26040d98f9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:22.722389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55485cd5-9630-4f7b-b691-26040d98f9ee None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:22.731305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:22.732874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:22.733374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:22.733871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] PATH_INFO: `/v3/projects/be8784249e65481f8b36e5a90f7b4fa6` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:22.735773 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] RBAC: Authorizing `identity:delete_project(project_id=be8784249e65481f8b36e5a90f7b4fa6)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:22.738408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:22.802467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be8784249e65481f8b36e5a90f7b4fa6', 'request_id': 'req-c2a4e958-00a2-4160-99f9-4d9b800b434e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:22.809890 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be8784249e65481f8b36e5a90f7b4fa6', 'request_id': 'req-c2a4e958-00a2-4160-99f9-4d9b800b434e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:22.816690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be8784249e65481f8b36e5a90f7b4fa6', 'request_id': 'req-c2a4e958-00a2-4160-99f9-4d9b800b434e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:22.831105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] The token cache is being invalidate because project be8784249e65481f8b36e5a90f7b4fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:22.831388 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:22.832068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:22.833654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 943/1890] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:22 2022] DELETE /identity/v3/projects/be8784249e65481f8b36e5a90f7b4fa6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:57:22.837984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fb86f-938c-490f-8c24-2f36d0cbd6ec service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:22.854067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fb86f-938c-490f-8c24-2f36d0cbd6ec service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:22.925656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:22.926994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:22.926994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:22.926994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] PATH_INFO: `/v3/users/5ffc17c902bf481d892a62b29744fd37` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:22.927979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ffc17c902bf481d892a62b29744fd37)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:22.929940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:22.995590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ffc17c902bf481d892a62b29744fd37'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:23.011445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ffc17c902bf481d892a62b29744fd37'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:23.019734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ffc17c902bf481d892a62b29744fd37'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:23.020650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ffc17c902bf481d892a62b29744fd37'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:23.031278 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-be0ae069-0ab7-4e50-aab8-cf6d12b07df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ffc17c902bf481d892a62b29744fd37'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:23.037906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 948/1891] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:22 2022] DELETE /identity/v3/users/5ffc17c902bf481d892a62b29744fd37 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:23.400277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:23.401748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:23.417894 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4e958-00a2-4160-99f9-4d9b800b434e admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:23.489666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:23.490530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:23.490843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:23.491118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] PATH_INFO: `/v3/projects/9d64fe71849f45afa1742c32d20689a8` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:23.492347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d64fe71849f45afa1742c32d20689a8)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:23.494257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:23.551290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d64fe71849f45afa1742c32d20689a8', 'request_id': 'req-495f331c-dffd-4c34-a749-e0f0c8f85035'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:23.558334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d64fe71849f45afa1742c32d20689a8', 'request_id': 'req-495f331c-dffd-4c34-a749-e0f0c8f85035'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:23.564559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d64fe71849f45afa1742c32d20689a8', 'request_id': 'req-495f331c-dffd-4c34-a749-e0f0c8f85035'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:23.582942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] The token cache is being invalidate because project 9d64fe71849f45afa1742c32d20689a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:23.583453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:23.584359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-495f331c-dffd-4c34-a749-e0f0c8f85035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:23.586140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 944/1892] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:23 2022] DELETE /identity/v3/projects/9d64fe71849f45afa1742c32d20689a8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:23.592602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ed7c7-502d-487b-82f8-5e9b730e58c7 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:23.593009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ed7c7-502d-487b-82f8-5e9b730e58c7 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:23.593335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ed7c7-502d-487b-82f8-5e9b730e58c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:23.649655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-ba3ed7c7-502d-487b-82f8-5e9b730e58c7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:23.651211 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ba3ed7c7-502d-487b-82f8-5e9b730e58c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:23.759098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 949/1893] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:23.769177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171576ea-35a5-4066-85bd-2559a9d7257f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:23.769468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171576ea-35a5-4066-85bd-2559a9d7257f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:23.769745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171576ea-35a5-4066-85bd-2559a9d7257f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:23.791844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-171576ea-35a5-4066-85bd-2559a9d7257f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:23.793911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-171576ea-35a5-4066-85bd-2559a9d7257f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:23.807501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 945/1894] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:57:23.812900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ed7c7-502d-487b-82f8-5e9b730e58c7 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:23.823656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ed7c7-502d-487b-82f8-5e9b730e58c7 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:23.831644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db53d74-a105-488d-b691-a90963fb17f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:23.832853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db53d74-a105-488d-b691-a90963fb17f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:23.833888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db53d74-a105-488d-b691-a90963fb17f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:23.834365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db53d74-a105-488d-b691-a90963fb17f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:23.835664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db53d74-a105-488d-b691-a90963fb17f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:23.836201 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db53d74-a105-488d-b691-a90963fb17f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:23.839102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db53d74-a105-488d-b691-a90963fb17f5 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:23.855332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 950/1895] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:57:23.862253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171576ea-35a5-4066-85bd-2559a9d7257f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:23.877710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171576ea-35a5-4066-85bd-2559a9d7257f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:23.882504 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979354-305b-4b0f-b2f2-4971a7998bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:23.883222 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979354-305b-4b0f-b2f2-4971a7998bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:23.883503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979354-305b-4b0f-b2f2-4971a7998bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:23.883785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979354-305b-4b0f-b2f2-4971a7998bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:23.884501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b979354-305b-4b0f-b2f2-4971a7998bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:23.886737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b979354-305b-4b0f-b2f2-4971a7998bb4 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:23.920328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 946/1896] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:23 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:23.925622 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db53d74-a105-488d-b691-a90963fb17f5 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:23.945211 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db53d74-a105-488d-b691-a90963fb17f5 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:24.047226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e22166-5e2f-4173-8587-782767d19520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:24.048416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e22166-5e2f-4173-8587-782767d19520 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.048929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e22166-5e2f-4173-8587-782767d19520 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.049438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e22166-5e2f-4173-8587-782767d19520 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.050807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e22166-5e2f-4173-8587-782767d19520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:24.054569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e22166-5e2f-4173-8587-782767d19520 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:24.084261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 951/1897] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:23 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:24.089674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979354-305b-4b0f-b2f2-4971a7998bb4 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:24.101353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979354-305b-4b0f-b2f2-4971a7998bb4 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:24.107232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26464386-1db8-4149-9684-1398875eff5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:24.108059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26464386-1db8-4149-9684-1398875eff5b admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.108349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26464386-1db8-4149-9684-1398875eff5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.108627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26464386-1db8-4149-9684-1398875eff5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.109378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26464386-1db8-4149-9684-1398875eff5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:24.111028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26464386-1db8-4149-9684-1398875eff5b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:24.126082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 947/1898] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:24.129855 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e22166-5e2f-4173-8587-782767d19520 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:24.142231 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e22166-5e2f-4173-8587-782767d19520 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:24.149108 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc56fa-24b2-4120-aceb-3899d37e7b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:24.150098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc56fa-24b2-4120-aceb-3899d37e7b7d admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.150466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc56fa-24b2-4120-aceb-3899d37e7b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.150900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc56fa-24b2-4120-aceb-3899d37e7b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.152017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc56fa-24b2-4120-aceb-3899d37e7b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:24.153917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc56fa-24b2-4120-aceb-3899d37e7b7d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:24.166890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 952/1899] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:24.171887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26464386-1db8-4149-9684-1398875eff5b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:24.186451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26464386-1db8-4149-9684-1398875eff5b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:24.194188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0408-a22c-46f2-92ca-19693fafa5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:24.195796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0408-a22c-46f2-92ca-19693fafa5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.196274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0408-a22c-46f2-92ca-19693fafa5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.196780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0408-a22c-46f2-92ca-19693fafa5a3 admin admin] PATH_INFO: `/v3/projects/8c976f2c0d604f498a6bd00ff495c7f8/users/ce83cf752d1a49d98cc76ee8ff51fb83/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.225590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd0408-a22c-46f2-92ca-19693fafa5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c976f2c0d604f498a6bd00ff495c7f8, user_id=ce83cf752d1a49d98cc76ee8ff51fb83, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:24.230851 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd0408-a22c-46f2-92ca-19693fafa5a3 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:24.249585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 948/1900] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:24 2022] PUT /identity/v3/projects/8c976f2c0d604f498a6bd00ff495c7f8/users/ce83cf752d1a49d98cc76ee8ff51fb83/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:24.267359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5082336-338c-4487-b58d-f41182f2a476 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.267727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5082336-338c-4487-b58d-f41182f2a476 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.268036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5082336-338c-4487-b58d-f41182f2a476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.298110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e5082336-338c-4487-b58d-f41182f2a476 None None] MFA Rules not processed for user `ce83cf752d1a49d98cc76ee8ff51fb83`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:24.299396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e5082336-338c-4487-b58d-f41182f2a476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:24.385161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 953/1901] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:24.390601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0408-a22c-46f2-92ca-19693fafa5a3 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:24.506642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0408-a22c-46f2-92ca-19693fafa5a3 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:24.513177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368825c6-e0a6-47bf-b970-5d3e698c0355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:24.514126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368825c6-e0a6-47bf-b970-5d3e698c0355 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.514628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368825c6-e0a6-47bf-b970-5d3e698c0355 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.515399 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368825c6-e0a6-47bf-b970-5d3e698c0355 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.516127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368825c6-e0a6-47bf-b970-5d3e698c0355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:24.518572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368825c6-e0a6-47bf-b970-5d3e698c0355 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:24.556126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 949/1902] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:24 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:24.560615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5082336-338c-4487-b58d-f41182f2a476 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:24.572176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5082336-338c-4487-b58d-f41182f2a476 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:24.638814 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058d47a-698a-4f0e-af54-ff2cd3ac3158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:24.639559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d47a-698a-4f0e-af54-ff2cd3ac3158 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.639840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d47a-698a-4f0e-af54-ff2cd3ac3158 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.640115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d47a-698a-4f0e-af54-ff2cd3ac3158 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.640836 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058d47a-698a-4f0e-af54-ff2cd3ac3158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:24.642776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058d47a-698a-4f0e-af54-ff2cd3ac3158 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:24.674562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 954/1903] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:24 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:57:24.679592 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368825c6-e0a6-47bf-b970-5d3e698c0355 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:24.699433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368825c6-e0a6-47bf-b970-5d3e698c0355 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:24.710238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0b2c4-44ad-4435-bb58-0c5732b0a309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:24.713187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0b2c4-44ad-4435-bb58-0c5732b0a309 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.713698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0b2c4-44ad-4435-bb58-0c5732b0a309 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.714192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0b2c4-44ad-4435-bb58-0c5732b0a309 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.716237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0b2c4-44ad-4435-bb58-0c5732b0a309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:24.718757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0b2c4-44ad-4435-bb58-0c5732b0a309 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:24.743549 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 950/1904] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:24.748250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058d47a-698a-4f0e-af54-ff2cd3ac3158 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:24.759443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058d47a-698a-4f0e-af54-ff2cd3ac3158 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:24.764275 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0563e-8d82-44d4-b186-32947df21fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:24.765179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0563e-8d82-44d4-b186-32947df21fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.765534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0563e-8d82-44d4-b186-32947df21fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.765900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0563e-8d82-44d4-b186-32947df21fb8 admin admin] PATH_INFO: `/v3/projects/4b0be48601b942b08d96c88f0d6ef528/users/ea90b30690c642cd91b60dd4bfd8a3da/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.784426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0563e-8d82-44d4-b186-32947df21fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0be48601b942b08d96c88f0d6ef528, user_id=ea90b30690c642cd91b60dd4bfd8a3da, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:24.787489 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0563e-8d82-44d4-b186-32947df21fb8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:24.803172 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 955/1905] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:24 2022] PUT /identity/v3/projects/4b0be48601b942b08d96c88f0d6ef528/users/ea90b30690c642cd91b60dd4bfd8a3da/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:24.807813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0b2c4-44ad-4435-bb58-0c5732b0a309 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:24.822073 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0b2c4-44ad-4435-bb58-0c5732b0a309 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:24.896199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb829d-c1c6-46c2-8126-a3a6cdbba082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:24.896807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb829d-c1c6-46c2-8126-a3a6cdbba082 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.897087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb829d-c1c6-46c2-8126-a3a6cdbba082 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.897367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb829d-c1c6-46c2-8126-a3a6cdbba082 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.898098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb829d-c1c6-46c2-8126-a3a6cdbba082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:24.899812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb829d-c1c6-46c2-8126-a3a6cdbba082 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:24.914928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 951/1906] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:24.920712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0563e-8d82-44d4-b186-32947df21fb8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:24.938915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0563e-8d82-44d4-b186-32947df21fb8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:24.944206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba407214-926b-4ad4-8a2d-07b6fa82dec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:24.945421 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba407214-926b-4ad4-8a2d-07b6fa82dec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.945791 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba407214-926b-4ad4-8a2d-07b6fa82dec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.946134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba407214-926b-4ad4-8a2d-07b6fa82dec9 admin admin] PATH_INFO: `/v3/projects/4b0be48601b942b08d96c88f0d6ef528/users/ea90b30690c642cd91b60dd4bfd8a3da/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:24.948436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba407214-926b-4ad4-8a2d-07b6fa82dec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0be48601b942b08d96c88f0d6ef528, user_id=ea90b30690c642cd91b60dd4bfd8a3da, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:24.954338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba407214-926b-4ad4-8a2d-07b6fa82dec9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:24.967851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 956/1907] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:24 2022] PUT /identity/v3/projects/4b0be48601b942b08d96c88f0d6ef528/users/ea90b30690c642cd91b60dd4bfd8a3da/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:24.985808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54da51e-cb03-472e-8066-e1a4963c91d2 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:24.986176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54da51e-cb03-472e-8066-e1a4963c91d2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:24.986696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54da51e-cb03-472e-8066-e1a4963c91d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:25.017142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c54da51e-cb03-472e-8066-e1a4963c91d2 None None] MFA Rules not processed for user `ea90b30690c642cd91b60dd4bfd8a3da`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:25.018384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c54da51e-cb03-472e-8066-e1a4963c91d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:25.100164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 952/1908] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:24 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:25.112516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba407214-926b-4ad4-8a2d-07b6fa82dec9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:25.114390 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ba407214-926b-4ad4-8a2d-07b6fa82dec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:25.178986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba407214-926b-4ad4-8a2d-07b6fa82dec9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:25.183396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357c7cb-c269-4eba-b838-1ebcd0144695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:25.184140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357c7cb-c269-4eba-b838-1ebcd0144695 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:25.184490 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357c7cb-c269-4eba-b838-1ebcd0144695 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:25.184770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357c7cb-c269-4eba-b838-1ebcd0144695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:25.195440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357c7cb-c269-4eba-b838-1ebcd0144695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:25.196929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357c7cb-c269-4eba-b838-1ebcd0144695 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:25.208359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 957/1909] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:25 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:57:27.052216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54da51e-cb03-472e-8066-e1a4963c91d2 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:27.095052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357c7cb-c269-4eba-b838-1ebcd0144695 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:27.097691 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1357c7cb-c269-4eba-b838-1ebcd0144695 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:27.181791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54da51e-cb03-472e-8066-e1a4963c91d2 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:27.188844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:27.189871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:27.191255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:27.191853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] PATH_INFO: `/v3/users/ce83cf752d1a49d98cc76ee8ff51fb83` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:27.194216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce83cf752d1a49d98cc76ee8ff51fb83)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:27.196774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:27.239430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357c7cb-c269-4eba-b838-1ebcd0144695 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:27.259033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:27.259954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:27.260299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:27.260614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] PATH_INFO: `/v3/users/0622aea8005b4b82bcb921089974775d` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:27.262045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0622aea8005b4b82bcb921089974775d)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:27.264943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:27.282289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce83cf752d1a49d98cc76ee8ff51fb83'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:27.283023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce83cf752d1a49d98cc76ee8ff51fb83'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:27.299992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce83cf752d1a49d98cc76ee8ff51fb83'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:27.300978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce83cf752d1a49d98cc76ee8ff51fb83'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:27.309673 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53ae9aba-3a6e-4f79-b585-f13562fac6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce83cf752d1a49d98cc76ee8ff51fb83'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:27.328764 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 953/1910] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:27 2022] DELETE /identity/v3/users/ce83cf752d1a49d98cc76ee8ff51fb83 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:27.332187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6a82f-7f45-4464-91b9-fc0efade5371 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:27.332187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6a82f-7f45-4464-91b9-fc0efade5371 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:27.332187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6a82f-7f45-4464-91b9-fc0efade5371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:27.360630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0622aea8005b4b82bcb921089974775d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:27.366886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-06f6a82f-7f45-4464-91b9-fc0efade5371 None None] MFA Rules not processed for user `8ef0eaa76ffe4b62a132f0f4ee595a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:27.368332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-06f6a82f-7f45-4464-91b9-fc0efade5371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:27.396894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0622aea8005b4b82bcb921089974775d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:27.410429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0622aea8005b4b82bcb921089974775d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:27.411672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0622aea8005b4b82bcb921089974775d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:27.425107 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bdb8387d-ae88-489d-a124-9a05e775888f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0622aea8005b4b82bcb921089974775d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:27.435020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 958/1911] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:27 2022] DELETE /identity/v3/users/0622aea8005b4b82bcb921089974775d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:27.436733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d26647-3a75-4489-8094-c8c14d6058b7 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:27.437017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d26647-3a75-4489-8094-c8c14d6058b7 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:27.437291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d26647-3a75-4489-8094-c8c14d6058b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:27.458519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 954/1912] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:27.475283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-d7d26647-3a75-4489-8094-c8c14d6058b7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:27.476525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d7d26647-3a75-4489-8094-c8c14d6058b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:27.574267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 959/1913] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:27.613265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6a82f-7f45-4464-91b9-fc0efade5371 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:27.616946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-06f6a82f-7f45-4464-91b9-fc0efade5371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:27.711003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6a82f-7f45-4464-91b9-fc0efade5371 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:27.800243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fd599-d165-4c1e-bd73-44ef92b47a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:27.800243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fd599-d165-4c1e-bd73-44ef92b47a21 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:27.800243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fd599-d165-4c1e-bd73-44ef92b47a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:27.800243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fd599-d165-4c1e-bd73-44ef92b47a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:27.817191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fd599-d165-4c1e-bd73-44ef92b47a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:27.819254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fd599-d165-4c1e-bd73-44ef92b47a21 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:27.835769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 955/1914] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:27.928918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d26647-3a75-4489-8094-c8c14d6058b7 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:27.931653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d7d26647-3a75-4489-8094-c8c14d6058b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:27.975560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d26647-3a75-4489-8094-c8c14d6058b7 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:27.985124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:28.000150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:28.000957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:28.002424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] PATH_INFO: `/v3/projects/d42ea16d0ee64f398b01b206c1960b08` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:28.005233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d42ea16d0ee64f398b01b206c1960b08)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:28.008947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:28.104313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd42ea16d0ee64f398b01b206c1960b08', 'request_id': 'req-81696503-c694-490d-9772-c50b71d9912c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:28.116760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd42ea16d0ee64f398b01b206c1960b08', 'request_id': 'req-81696503-c694-490d-9772-c50b71d9912c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:28.132908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd42ea16d0ee64f398b01b206c1960b08', 'request_id': 'req-81696503-c694-490d-9772-c50b71d9912c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:28.162242 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] The token cache is being invalidate because project d42ea16d0ee64f398b01b206c1960b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:28.162972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:28.163930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-81696503-c694-490d-9772-c50b71d9912c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:28.168433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 960/1915] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:27 2022] DELETE /identity/v3/projects/d42ea16d0ee64f398b01b206c1960b08 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:28.175054 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23ba68-f8b6-47c1-ba61-3b017603961a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:28.175671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23ba68-f8b6-47c1-ba61-3b017603961a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:28.175819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23ba68-f8b6-47c1-ba61-3b017603961a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:28.213208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-6a23ba68-f8b6-47c1-ba61-3b017603961a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:28.215791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6a23ba68-f8b6-47c1-ba61-3b017603961a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:28.369449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 956/1916] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:57:28.384780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1a26-6e36-4f1c-94b3-420be1ad2927 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:28.385379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1a26-6e36-4f1c-94b3-420be1ad2927 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:28.385890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1a26-6e36-4f1c-94b3-420be1ad2927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:28.425086 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-868f1a26-6e36-4f1c-94b3-420be1ad2927 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:28.427707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-868f1a26-6e36-4f1c-94b3-420be1ad2927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:28.454392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 961/1917] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:28.459532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23ba68-f8b6-47c1-ba61-3b017603961a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:28.474927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23ba68-f8b6-47c1-ba61-3b017603961a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:28.483876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6deab8-664d-4b50-9fd4-e941a71a65a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:28.484829 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6deab8-664d-4b50-9fd4-e941a71a65a7 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:28.485208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6deab8-664d-4b50-9fd4-e941a71a65a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:28.485752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6deab8-664d-4b50-9fd4-e941a71a65a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:28.487028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6deab8-664d-4b50-9fd4-e941a71a65a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:28.487725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6deab8-664d-4b50-9fd4-e941a71a65a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:28.493012 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6deab8-664d-4b50-9fd4-e941a71a65a7 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:28.509188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 957/1918] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:57:28.513810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f1a26-6e36-4f1c-94b3-420be1ad2927 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:28.526585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f1a26-6e36-4f1c-94b3-420be1ad2927 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:28.532937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b00db-2c2a-47ee-903f-71ff4c53cad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:28.533926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b00db-2c2a-47ee-903f-71ff4c53cad3 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:28.534411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b00db-2c2a-47ee-903f-71ff4c53cad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:28.534909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b00db-2c2a-47ee-903f-71ff4c53cad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:28.535802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b00db-2c2a-47ee-903f-71ff4c53cad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:28.538080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b00db-2c2a-47ee-903f-71ff4c53cad3 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:28.560367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6deab8-664d-4b50-9fd4-e941a71a65a7 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:28.562597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9d6deab8-664d-4b50-9fd4-e941a71a65a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:28.592795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 962/1919] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:28 2022] POST /identity/v3/projects => generated 365 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:28.593871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b00db-2c2a-47ee-903f-71ff4c53cad3 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:28.594688 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6deab8-664d-4b50-9fd4-e941a71a65a7 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:28.680477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:28.681630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:28.681910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:28.682176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] PATH_INFO: `/v3/projects/8c976f2c0d604f498a6bd00ff495c7f8` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:28.684284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c976f2c0d604f498a6bd00ff495c7f8)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:28.686882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:28.691492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b00db-2c2a-47ee-903f-71ff4c53cad3 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:28.698173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50fad0-980d-44ca-8a1f-fd1a1f8e9bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:28.699223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50fad0-980d-44ca-8a1f-fd1a1f8e9bae admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:28.700498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50fad0-980d-44ca-8a1f-fd1a1f8e9bae admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:28.700993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50fad0-980d-44ca-8a1f-fd1a1f8e9bae admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:28.701952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50fad0-980d-44ca-8a1f-fd1a1f8e9bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:28.703959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50fad0-980d-44ca-8a1f-fd1a1f8e9bae admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:28.728353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 963/1920] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:28 2022] POST /identity/v3/users => generated 283 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:28.734485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50fad0-980d-44ca-8a1f-fd1a1f8e9bae admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:28.751143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50fad0-980d-44ca-8a1f-fd1a1f8e9bae admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:28.757102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c976f2c0d604f498a6bd00ff495c7f8', 'request_id': 'req-3116cefc-d699-448e-b810-01853f03a898'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:28.770738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c976f2c0d604f498a6bd00ff495c7f8', 'request_id': 'req-3116cefc-d699-448e-b810-01853f03a898'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:28.779313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c976f2c0d604f498a6bd00ff495c7f8', 'request_id': 'req-3116cefc-d699-448e-b810-01853f03a898'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:28.809976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] The token cache is being invalidate because project 8c976f2c0d604f498a6bd00ff495c7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:28.810531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:28.812660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:28.822646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 958/1921] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:28 2022] DELETE /identity/v3/projects/8c976f2c0d604f498a6bd00ff495c7f8 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:28.828563 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:28.831869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:28.872629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612a2dc-97ef-4802-ab7c-8d6a3e8bb935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:28.874347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612a2dc-97ef-4802-ab7c-8d6a3e8bb935 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:28.874748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612a2dc-97ef-4802-ab7c-8d6a3e8bb935 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:28.875103 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612a2dc-97ef-4802-ab7c-8d6a3e8bb935 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:28.875954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612a2dc-97ef-4802-ab7c-8d6a3e8bb935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:28.877969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612a2dc-97ef-4802-ab7c-8d6a3e8bb935 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:28.905433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 964/1922] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:57:28.919799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612a2dc-97ef-4802-ab7c-8d6a3e8bb935 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:28.924698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-4612a2dc-97ef-4802-ab7c-8d6a3e8bb935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:28.942020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612a2dc-97ef-4802-ab7c-8d6a3e8bb935 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:28.952682 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116cefc-d699-448e-b810-01853f03a898 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:28.953289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ffdf7-4cc7-4d17-8ed5-d62652c60610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:28.954008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ffdf7-4cc7-4d17-8ed5-d62652c60610 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:28.954620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ffdf7-4cc7-4d17-8ed5-d62652c60610 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:28.955261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ffdf7-4cc7-4d17-8ed5-d62652c60610 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:28.956430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ffdf7-4cc7-4d17-8ed5-d62652c60610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:28.958574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ffdf7-4cc7-4d17-8ed5-d62652c60610 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:28.958986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:28.959961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:28.960245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:28.960727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] PATH_INFO: `/v3/users/ea90b30690c642cd91b60dd4bfd8a3da` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:28.962694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea90b30690c642cd91b60dd4bfd8a3da)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:28.964605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:28.981837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 965/1923] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:28.987749 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ffdf7-4cc7-4d17-8ed5-d62652c60610 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:29.002259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ffdf7-4cc7-4d17-8ed5-d62652c60610 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:29.012058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e0d3e-ebba-441f-b748-d0f1e5e092cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:29.014049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0d3e-ebba-441f-b748-d0f1e5e092cf admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:29.014562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0d3e-ebba-441f-b748-d0f1e5e092cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:29.015131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e0d3e-ebba-441f-b748-d0f1e5e092cf admin admin] PATH_INFO: `/v3/projects/9e16f3e5dc0046fdb80c08e3d4853b24/users/dc2aff3e556549e2b6f0de51623674b4/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:29.040467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e0d3e-ebba-441f-b748-d0f1e5e092cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e16f3e5dc0046fdb80c08e3d4853b24, user_id=dc2aff3e556549e2b6f0de51623674b4, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:29.043687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e0d3e-ebba-441f-b748-d0f1e5e092cf admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:29.050651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea90b30690c642cd91b60dd4bfd8a3da'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:29.053737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea90b30690c642cd91b60dd4bfd8a3da'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:29.064314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 966/1924] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:28 2022] PUT /identity/v3/projects/9e16f3e5dc0046fdb80c08e3d4853b24/users/dc2aff3e556549e2b6f0de51623674b4/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:29.076517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea90b30690c642cd91b60dd4bfd8a3da'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:29.077749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea90b30690c642cd91b60dd4bfd8a3da'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:29.079462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb6f7e-d051-4194-8fbd-c4e003b12f4d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:29.079875 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb6f7e-d051-4194-8fbd-c4e003b12f4d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:29.080249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb6f7e-d051-4194-8fbd-c4e003b12f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:29.090637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea90b30690c642cd91b60dd4bfd8a3da'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:29.109532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-72bb6f7e-d051-4194-8fbd-c4e003b12f4d None None] MFA Rules not processed for user `dc2aff3e556549e2b6f0de51623674b4`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:29.110793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-72bb6f7e-d051-4194-8fbd-c4e003b12f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:29.111268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 959/1925] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:28 2022] DELETE /identity/v3/users/ea90b30690c642cd91b60dd4bfd8a3da => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:29.213583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 967/1926] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:57:29.227802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:29.230160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:29.310417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cb977-ecb0-47e3-a98d-2626e38749dd admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:29.315571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfda33-478e-4cea-8136-7b1bb61cec1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:29.316252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfda33-478e-4cea-8136-7b1bb61cec1b service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:29.316544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfda33-478e-4cea-8136-7b1bb61cec1b service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:29.316874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfda33-478e-4cea-8136-7b1bb61cec1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:29.329000 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfda33-478e-4cea-8136-7b1bb61cec1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:29.330570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfda33-478e-4cea-8136-7b1bb61cec1b service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:29.346496 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 960/1927] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:29.473913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb6f7e-d051-4194-8fbd-c4e003b12f4d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:29.476003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-72bb6f7e-d051-4194-8fbd-c4e003b12f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:29.593047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb6f7e-d051-4194-8fbd-c4e003b12f4d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:29.598113 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:29.599012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:29.599288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:29.599561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] PATH_INFO: `/v3/projects/4b0be48601b942b08d96c88f0d6ef528` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:29.600801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b0be48601b942b08d96c88f0d6ef528)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:29.602758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:29.667792 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b0be48601b942b08d96c88f0d6ef528', 'request_id': 'req-583a4c26-ec5d-4813-94f0-658c7416bd7a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:29.675689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b0be48601b942b08d96c88f0d6ef528', 'request_id': 'req-583a4c26-ec5d-4813-94f0-658c7416bd7a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:29.689187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b0be48601b942b08d96c88f0d6ef528', 'request_id': 'req-583a4c26-ec5d-4813-94f0-658c7416bd7a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:29.714137 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] The token cache is being invalidate because project 4b0be48601b942b08d96c88f0d6ef528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:29.714747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:29.716310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-583a4c26-ec5d-4813-94f0-658c7416bd7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:29.718430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 968/1928] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:29 2022] DELETE /identity/v3/projects/4b0be48601b942b08d96c88f0d6ef528 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:29.725192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352470-38eb-4510-af6e-b3947a0660b7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:29.725614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352470-38eb-4510-af6e-b3947a0660b7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:29.726005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352470-38eb-4510-af6e-b3947a0660b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:29.751803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1c352470-38eb-4510-af6e-b3947a0660b7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:29.753471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1c352470-38eb-4510-af6e-b3947a0660b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:29.843871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 961/1929] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:29.862504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fabcf1-fcb3-456c-b563-98f83a9eb451 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:29.863148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fabcf1-fcb3-456c-b563-98f83a9eb451 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:29.863663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fabcf1-fcb3-456c-b563-98f83a9eb451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:29.893148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a2fabcf1-fcb3-456c-b563-98f83a9eb451 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:29.894951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a2fabcf1-fcb3-456c-b563-98f83a9eb451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:29.904931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 969/1930] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:29.909060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c352470-38eb-4510-af6e-b3947a0660b7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:29.919905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c352470-38eb-4510-af6e-b3947a0660b7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:29.925376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75240f27-1f2f-4a25-bd27-03b84fbc58ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:29.926228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75240f27-1f2f-4a25-bd27-03b84fbc58ab admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:29.926610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75240f27-1f2f-4a25-bd27-03b84fbc58ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:29.927023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75240f27-1f2f-4a25-bd27-03b84fbc58ab admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:29.927884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75240f27-1f2f-4a25-bd27-03b84fbc58ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:29.928441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75240f27-1f2f-4a25-bd27-03b84fbc58ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:29.930234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75240f27-1f2f-4a25-bd27-03b84fbc58ab admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:29.955612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 962/1931] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:57:29.960344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fabcf1-fcb3-456c-b563-98f83a9eb451 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:29.970949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fabcf1-fcb3-456c-b563-98f83a9eb451 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:29.987107 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fede099-836f-4bb3-b929-082aa96c8d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:29.987823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fede099-836f-4bb3-b929-082aa96c8d77 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:29.988317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fede099-836f-4bb3-b929-082aa96c8d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:29.989377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fede099-836f-4bb3-b929-082aa96c8d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:29.990309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fede099-836f-4bb3-b929-082aa96c8d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:29.993119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fede099-836f-4bb3-b929-082aa96c8d77 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:30.050058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 970/1932] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:29 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:30.052525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75240f27-1f2f-4a25-bd27-03b84fbc58ab admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:30.110082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75240f27-1f2f-4a25-bd27-03b84fbc58ab admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:30.259165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8538f4-467a-494b-8139-8aa0c992fd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:30.261095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8538f4-467a-494b-8139-8aa0c992fd73 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:30.261095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8538f4-467a-494b-8139-8aa0c992fd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:30.261553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8538f4-467a-494b-8139-8aa0c992fd73 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:30.263278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8538f4-467a-494b-8139-8aa0c992fd73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:30.266988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8538f4-467a-494b-8139-8aa0c992fd73 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:30.301543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 963/1933] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:30 2022] POST /identity/v3/users => generated 291 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:30.308713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fede099-836f-4bb3-b929-082aa96c8d77 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:30.328815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fede099-836f-4bb3-b929-082aa96c8d77 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:30.338668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec8840-5155-4d72-ae5e-a2386acc23e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:30.339940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec8840-5155-4d72-ae5e-a2386acc23e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:30.340535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec8840-5155-4d72-ae5e-a2386acc23e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:30.341067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec8840-5155-4d72-ae5e-a2386acc23e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:30.342564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec8840-5155-4d72-ae5e-a2386acc23e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:30.345827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec8840-5155-4d72-ae5e-a2386acc23e8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:30.367618 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 971/1934] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:30.372832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8538f4-467a-494b-8139-8aa0c992fd73 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:30.383820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8538f4-467a-494b-8139-8aa0c992fd73 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:30.388816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fb07f-aa59-496d-a83a-0f1b3cc5e148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:30.389493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fb07f-aa59-496d-a83a-0f1b3cc5e148 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:30.389793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fb07f-aa59-496d-a83a-0f1b3cc5e148 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:30.390120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fb07f-aa59-496d-a83a-0f1b3cc5e148 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:30.390877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fb07f-aa59-496d-a83a-0f1b3cc5e148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:30.392493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fb07f-aa59-496d-a83a-0f1b3cc5e148 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:30.408557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 964/1935] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:30.411794 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec8840-5155-4d72-ae5e-a2386acc23e8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:30.428986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec8840-5155-4d72-ae5e-a2386acc23e8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:30.438265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eed4a4-6479-493d-bf53-7b64ee140ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:30.440687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eed4a4-6479-493d-bf53-7b64ee140ead admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:30.441215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eed4a4-6479-493d-bf53-7b64ee140ead admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:30.441660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eed4a4-6479-493d-bf53-7b64ee140ead admin admin] PATH_INFO: `/v3/projects/354352f72c3a49a6893e809c63a55d95/users/d82dc0f11c0343e7ab6bd376a1d5800a/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:30.463937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eed4a4-6479-493d-bf53-7b64ee140ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=354352f72c3a49a6893e809c63a55d95, user_id=d82dc0f11c0343e7ab6bd376a1d5800a, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:30.467326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eed4a4-6479-493d-bf53-7b64ee140ead admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:30.489144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 972/1936] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:30 2022] PUT /identity/v3/projects/354352f72c3a49a6893e809c63a55d95/users/d82dc0f11c0343e7ab6bd376a1d5800a/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:30.507164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8ac6-bd3e-4d92-8576-c2d0c7c113d5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:30.507543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8ac6-bd3e-4d92-8576-c2d0c7c113d5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:30.508156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c8ac6-bd3e-4d92-8576-c2d0c7c113d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:30.549147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-bf4c8ac6-bd3e-4d92-8576-c2d0c7c113d5 None None] MFA Rules not processed for user `d82dc0f11c0343e7ab6bd376a1d5800a`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:30.551144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-bf4c8ac6-bd3e-4d92-8576-c2d0c7c113d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:30.583370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eed4a4-6479-493d-bf53-7b64ee140ead admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:30.587045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a0eed4a4-6479-493d-bf53-7b64ee140ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:30.662500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 965/1937] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:30.667537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c8ac6-bd3e-4d92-8576-c2d0c7c113d5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:30.687826 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eed4a4-6479-493d-bf53-7b64ee140ead admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:30.695521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:30.696530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:30.696950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:30.697358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] PATH_INFO: `/v3/users/ee7aeeef29a2456ca21d74d27ec08167` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:30.699450 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee7aeeef29a2456ca21d74d27ec08167)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:30.702608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:30.774604 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee7aeeef29a2456ca21d74d27ec08167'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:30.782810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c8ac6-bd3e-4d92-8576-c2d0c7c113d5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:30.788370 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c3e4f-2f39-42ce-bd1f-54ad58791722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:30.789101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c3e4f-2f39-42ce-bd1f-54ad58791722 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:30.789329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c3e4f-2f39-42ce-bd1f-54ad58791722 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:30.789608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c3e4f-2f39-42ce-bd1f-54ad58791722 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:30.790965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c3e4f-2f39-42ce-bd1f-54ad58791722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:30.792933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c3e4f-2f39-42ce-bd1f-54ad58791722 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:30.804008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee7aeeef29a2456ca21d74d27ec08167'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:30.818579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee7aeeef29a2456ca21d74d27ec08167'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:30.820070 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee7aeeef29a2456ca21d74d27ec08167'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:30.829616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 966/1938] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:30 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:57:30.834861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c3e4f-2f39-42ce-bd1f-54ad58791722 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:30.836311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f0c6c9-1bf5-41b0-8a91-627035a24bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee7aeeef29a2456ca21d74d27ec08167'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:30.848268 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 973/1939] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:30 2022] DELETE /identity/v3/users/ee7aeeef29a2456ca21d74d27ec08167 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:30.854929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c4f8-5249-4f09-b78b-4054c28fecaf None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:30.856897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c4f8-5249-4f09-b78b-4054c28fecaf None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:30.857462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c4f8-5249-4f09-b78b-4054c28fecaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:30.893915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2089c4f8-5249-4f09-b78b-4054c28fecaf None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:30.896388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2089c4f8-5249-4f09-b78b-4054c28fecaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:30.953007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c3e4f-2f39-42ce-bd1f-54ad58791722 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:31.071213 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 974/1940] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:57:31.077990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df379d7-7f7d-4489-8ba0-b972a6d59717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:31.078696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df379d7-7f7d-4489-8ba0-b972a6d59717 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:31.079011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df379d7-7f7d-4489-8ba0-b972a6d59717 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:31.079287 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df379d7-7f7d-4489-8ba0-b972a6d59717 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:31.080047 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df379d7-7f7d-4489-8ba0-b972a6d59717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:31.082080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df379d7-7f7d-4489-8ba0-b972a6d59717 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:31.108233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089c4f8-5249-4f09-b78b-4054c28fecaf None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:31.112179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2089c4f8-5249-4f09-b78b-4054c28fecaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:31.116828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 967/1941] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:30 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:31.122182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df379d7-7f7d-4489-8ba0-b972a6d59717 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:31.126338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5df379d7-7f7d-4489-8ba0-b972a6d59717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:31.162371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df379d7-7f7d-4489-8ba0-b972a6d59717 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:31.170145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a8ae8-7c4d-485b-a646-f80f73b6198c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:31.172016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a8ae8-7c4d-485b-a646-f80f73b6198c admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:31.173004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a8ae8-7c4d-485b-a646-f80f73b6198c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:31.173967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a8ae8-7c4d-485b-a646-f80f73b6198c admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:31.175700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a8ae8-7c4d-485b-a646-f80f73b6198c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:31.180588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a8ae8-7c4d-485b-a646-f80f73b6198c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:31.202169 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 968/1942] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:31.206990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a8ae8-7c4d-485b-a646-f80f73b6198c admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:31.222440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a8ae8-7c4d-485b-a646-f80f73b6198c admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:31.222677 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089c4f8-5249-4f09-b78b-4054c28fecaf None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:31.227625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d454954-cdcb-4e0f-865b-d8befef5c4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:31.228393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454954-cdcb-4e0f-865b-d8befef5c4bd service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:31.228784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454954-cdcb-4e0f-865b-d8befef5c4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:31.229244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454954-cdcb-4e0f-865b-d8befef5c4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:31.230118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0cf59-d1d6-42b9-92bd-52e12ae89312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:31.233812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0cf59-d1d6-42b9-92bd-52e12ae89312 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:31.234643 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0cf59-d1d6-42b9-92bd-52e12ae89312 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:31.235972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0cf59-d1d6-42b9-92bd-52e12ae89312 admin admin] PATH_INFO: `/v3/projects/89c32586514b4e30b8cccde71fd9812e/users/d4cda3c1d52c4c61860da5e34aa3b1df/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:31.248382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d454954-cdcb-4e0f-865b-d8befef5c4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:31.252333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d454954-cdcb-4e0f-865b-d8befef5c4bd service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:31.260991 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0cf59-d1d6-42b9-92bd-52e12ae89312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c32586514b4e30b8cccde71fd9812e, user_id=d4cda3c1d52c4c61860da5e34aa3b1df, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:31.270742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0cf59-d1d6-42b9-92bd-52e12ae89312 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:31.276007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 975/1943] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:31.289358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 969/1944] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:31 2022] PUT /identity/v3/projects/89c32586514b4e30b8cccde71fd9812e/users/d4cda3c1d52c4c61860da5e34aa3b1df/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:57:31.294004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d454954-cdcb-4e0f-865b-d8befef5c4bd service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:31.312359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d454954-cdcb-4e0f-865b-d8befef5c4bd service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:31.426829 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e2b96-0cdf-4aa3-ab07-939ac318f9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:31.427943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e2b96-0cdf-4aa3-ab07-939ac318f9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:31.428438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e2b96-0cdf-4aa3-ab07-939ac318f9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:31.428898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e2b96-0cdf-4aa3-ab07-939ac318f9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:31.430334 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e2b96-0cdf-4aa3-ab07-939ac318f9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:31.432673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e2b96-0cdf-4aa3-ab07-939ac318f9e9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:31.458774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 976/1945] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:57:31.462898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0cf59-d1d6-42b9-92bd-52e12ae89312 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:31.480066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0cf59-d1d6-42b9-92bd-52e12ae89312 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:31.486866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f2a22-262b-4e4b-9a2e-b386af7701bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:31.488512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f2a22-262b-4e4b-9a2e-b386af7701bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:31.488994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f2a22-262b-4e4b-9a2e-b386af7701bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:31.489369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f2a22-262b-4e4b-9a2e-b386af7701bc admin admin] PATH_INFO: `/v3/projects/89c32586514b4e30b8cccde71fd9812e/users/d4cda3c1d52c4c61860da5e34aa3b1df/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:31.492152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f2a22-262b-4e4b-9a2e-b386af7701bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c32586514b4e30b8cccde71fd9812e, user_id=d4cda3c1d52c4c61860da5e34aa3b1df, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:31.495098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f2a22-262b-4e4b-9a2e-b386af7701bc admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:31.513113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 970/1946] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:31 2022] PUT /identity/v3/projects/89c32586514b4e30b8cccde71fd9812e/users/d4cda3c1d52c4c61860da5e34aa3b1df/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:31.527263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20822ee5-9854-4850-914c-ac669b7f8ca0 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:31.527571 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20822ee5-9854-4850-914c-ac669b7f8ca0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:31.527872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20822ee5-9854-4850-914c-ac669b7f8ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:31.565364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-20822ee5-9854-4850-914c-ac669b7f8ca0 None None] MFA Rules not processed for user `d4cda3c1d52c4c61860da5e34aa3b1df`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:31.567046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-20822ee5-9854-4850-914c-ac669b7f8ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:31.688146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 977/1947] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:31.703462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f2a22-262b-4e4b-9a2e-b386af7701bc admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:31.706467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-634f2a22-262b-4e4b-9a2e-b386af7701bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:31.950912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f2a22-262b-4e4b-9a2e-b386af7701bc admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:31.961138 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771de7b2-14f1-4922-936d-307fcbcd95b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:31.962315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de7b2-14f1-4922-936d-307fcbcd95b4 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:31.962976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de7b2-14f1-4922-936d-307fcbcd95b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:31.963560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de7b2-14f1-4922-936d-307fcbcd95b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:31.979542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771de7b2-14f1-4922-936d-307fcbcd95b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:31.982130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771de7b2-14f1-4922-936d-307fcbcd95b4 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:31.993800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20822ee5-9854-4850-914c-ac669b7f8ca0 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:31.998182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-20822ee5-9854-4850-914c-ac669b7f8ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:32.015911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 971/1948] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:32.163020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20822ee5-9854-4850-914c-ac669b7f8ca0 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:32.175419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:32.176905 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:32.177457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:32.177918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] PATH_INFO: `/v3/projects/eef6814e331548b092b90ddcf3665b66` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:32.180379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef6814e331548b092b90ddcf3665b66)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:32.190608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:32.288952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef6814e331548b092b90ddcf3665b66', 'request_id': 'req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:32.306513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef6814e331548b092b90ddcf3665b66', 'request_id': 'req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:32.322806 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef6814e331548b092b90ddcf3665b66', 'request_id': 'req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:32.352017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] The token cache is being invalidate because project eef6814e331548b092b90ddcf3665b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:32.354146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:32.355335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-cfe2fdb8-6f1c-45cb-afd3-a79acefe2adb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:32.358029 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 978/1949] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:31 2022] DELETE /identity/v3/projects/eef6814e331548b092b90ddcf3665b66 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:57:32.364387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed1d7a-fe43-434a-b7af-921f0c57dd8a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:32.365174 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed1d7a-fe43-434a-b7af-921f0c57dd8a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:32.365174 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed1d7a-fe43-434a-b7af-921f0c57dd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:32.409114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b6ed1d7a-fe43-434a-b7af-921f0c57dd8a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:32.413243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b6ed1d7a-fe43-434a-b7af-921f0c57dd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:32.564420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 972/1950] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:57:32.581988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6dc1a-3983-4acb-997c-90e2af6b6880 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:32.582381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6dc1a-3983-4acb-997c-90e2af6b6880 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:32.583140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6dc1a-3983-4acb-997c-90e2af6b6880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:32.624402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0fb6dc1a-3983-4acb-997c-90e2af6b6880 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:32.625664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0fb6dc1a-3983-4acb-997c-90e2af6b6880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:32.652419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed1d7a-fe43-434a-b7af-921f0c57dd8a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:32.652634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 979/1951] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:32.666525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed1d7a-fe43-434a-b7af-921f0c57dd8a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:32.675749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef56453-a936-4d50-a53e-cd05960bb530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:32.676704 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef56453-a936-4d50-a53e-cd05960bb530 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:32.677351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef56453-a936-4d50-a53e-cd05960bb530 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:32.678026 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef56453-a936-4d50-a53e-cd05960bb530 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:32.679821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef56453-a936-4d50-a53e-cd05960bb530 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:32.680317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef56453-a936-4d50-a53e-cd05960bb530 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:32.682418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef56453-a936-4d50-a53e-cd05960bb530 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:32.696680 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6dc1a-3983-4acb-997c-90e2af6b6880 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:32.698332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0fb6dc1a-3983-4acb-997c-90e2af6b6880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:32.708424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 973/1952] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:57:32.717442 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef56453-a936-4d50-a53e-cd05960bb530 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:32.741878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef56453-a936-4d50-a53e-cd05960bb530 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:32.761328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61166838-0adc-4516-bb1b-00e5254fef9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:32.762020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61166838-0adc-4516-bb1b-00e5254fef9a admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:32.762919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61166838-0adc-4516-bb1b-00e5254fef9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:32.763198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61166838-0adc-4516-bb1b-00e5254fef9a admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:32.763913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61166838-0adc-4516-bb1b-00e5254fef9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:32.765901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61166838-0adc-4516-bb1b-00e5254fef9a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:32.830128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 974/1953] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:32 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:32.835605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61166838-0adc-4516-bb1b-00e5254fef9a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:32.875083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6dc1a-3983-4acb-997c-90e2af6b6880 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:32.981477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61166838-0adc-4516-bb1b-00e5254fef9a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:32.993079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2b1c4-a5bc-49d2-8f3a-b1123064e054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:32.994573 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2b1c4-a5bc-49d2-8f3a-b1123064e054 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:32.995409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2b1c4-a5bc-49d2-8f3a-b1123064e054 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:32.995950 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2b1c4-a5bc-49d2-8f3a-b1123064e054 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:32.997918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2b1c4-a5bc-49d2-8f3a-b1123064e054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:33.002686 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2b1c4-a5bc-49d2-8f3a-b1123064e054 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:33.043066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 975/1954] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:32 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:33.043701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923fea1-db22-4801-b547-4a418aa22aae tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4cda3c1d52c4c61860da5e34aa3b1df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89c32586514b4e30b8cccde71fd9812e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:33.044871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923fea1-db22-4801-b547-4a418aa22aae tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.045648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923fea1-db22-4801-b547-4a418aa22aae tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.046140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923fea1-db22-4801-b547-4a418aa22aae tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:33.047199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923fea1-db22-4801-b547-4a418aa22aae tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:33.047816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923fea1-db22-4801-b547-4a418aa22aae tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:33.049495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2b1c4-a5bc-49d2-8f3a-b1123064e054 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:33.049980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923fea1-db22-4801-b547-4a418aa22aae tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:33.068903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2b1c4-a5bc-49d2-8f3a-b1123064e054 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:33.073999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 980/1955] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:57:33.076064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923fea1-db22-4801-b547-4a418aa22aae tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:33.076930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be1e25-cb2a-41f5-a763-c8b0dd90e1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:33.076930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be1e25-cb2a-41f5-a763-c8b0dd90e1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.077245 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be1e25-cb2a-41f5-a763-c8b0dd90e1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.077394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be1e25-cb2a-41f5-a763-c8b0dd90e1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:33.078081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be1e25-cb2a-41f5-a763-c8b0dd90e1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:33.080173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be1e25-cb2a-41f5-a763-c8b0dd90e1c6 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:33.092622 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923fea1-db22-4801-b547-4a418aa22aae tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:33.099191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0793c2-ac7c-4295-bfb2-20338979fa3e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4cda3c1d52c4c61860da5e34aa3b1df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89c32586514b4e30b8cccde71fd9812e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:33.100137 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0793c2-ac7c-4295-bfb2-20338979fa3e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.100714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0793c2-ac7c-4295-bfb2-20338979fa3e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.101173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0793c2-ac7c-4295-bfb2-20338979fa3e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:33.101850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 976/1956] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:33.102462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0793c2-ac7c-4295-bfb2-20338979fa3e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:33.105704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0793c2-ac7c-4295-bfb2-20338979fa3e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:33.114230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be1e25-cb2a-41f5-a763-c8b0dd90e1c6 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:33.130396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be1e25-cb2a-41f5-a763-c8b0dd90e1c6 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:33.139081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a000b-5947-441e-8413-912c4dcaae08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:33.140365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a000b-5947-441e-8413-912c4dcaae08 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.141028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a000b-5947-441e-8413-912c4dcaae08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.141485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a000b-5947-441e-8413-912c4dcaae08 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:33.142431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a000b-5947-441e-8413-912c4dcaae08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:33.142737 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 981/1957] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:33 2022] POST /identity/v3/projects => generated 353 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:33.145504 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a000b-5947-441e-8413-912c4dcaae08 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:33.147785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0793c2-ac7c-4295-bfb2-20338979fa3e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:33.165536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 977/1958] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:57:33.171139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a000b-5947-441e-8413-912c4dcaae08 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:33.188798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a000b-5947-441e-8413-912c4dcaae08 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:33.241498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0793c2-ac7c-4295-bfb2-20338979fa3e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:33.251124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b05317-af30-4512-8cc6-0548fb67b10e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4cda3c1d52c4c61860da5e34aa3b1df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89c32586514b4e30b8cccde71fd9812e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:33.252308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b05317-af30-4512-8cc6-0548fb67b10e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.252767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b05317-af30-4512-8cc6-0548fb67b10e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.253220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b05317-af30-4512-8cc6-0548fb67b10e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:33.255625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b05317-af30-4512-8cc6-0548fb67b10e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:33.255625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b05317-af30-4512-8cc6-0548fb67b10e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:33.257610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b05317-af30-4512-8cc6-0548fb67b10e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:33.276817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 982/1959] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:57:33.294766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e165f-8fe7-45e4-8b03-679e3c4f20f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:33.296772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e165f-8fe7-45e4-8b03-679e3c4f20f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.297121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e165f-8fe7-45e4-8b03-679e3c4f20f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.297396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e165f-8fe7-45e4-8b03-679e3c4f20f2 admin admin] PATH_INFO: `/v3/projects/a6843d24fbf2449cbd117b8d9cfc7072/users/902bea79f26a4c1d8c3c1937f99f9d4b/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:33.327349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e165f-8fe7-45e4-8b03-679e3c4f20f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6843d24fbf2449cbd117b8d9cfc7072, user_id=902bea79f26a4c1d8c3c1937f99f9d4b, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:33.327523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b05317-af30-4512-8cc6-0548fb67b10e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:33.328206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e165f-8fe7-45e4-8b03-679e3c4f20f2 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:33.337915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b05317-af30-4512-8cc6-0548fb67b10e tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:33.344381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4cda3c1d52c4c61860da5e34aa3b1df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89c32586514b4e30b8cccde71fd9812e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:33.345283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.345566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.345847 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] PATH_INFO: `/v3/projects/9e18dbeb2f134edba56094a9ee6faa7b` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:33.347159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9e18dbeb2f134edba56094a9ee6faa7b)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:33.348058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 978/1960] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:33 2022] PUT /identity/v3/projects/a6843d24fbf2449cbd117b8d9cfc7072/users/902bea79f26a4c1d8c3c1937f99f9d4b/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:57:33.349312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:33.353909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76206cc-5ac9-4f20-afba-e06f06580ea7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.354215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76206cc-5ac9-4f20-afba-e06f06580ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.354514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76206cc-5ac9-4f20-afba-e06f06580ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:33.376273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-d76206cc-5ac9-4f20-afba-e06f06580ea7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:33.378128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d76206cc-5ac9-4f20-afba-e06f06580ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:33.427424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e18dbeb2f134edba56094a9ee6faa7b', 'request_id': 'req-c9051cfe-cdfb-4a5f-8598-7f829678183c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:33.435120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e18dbeb2f134edba56094a9ee6faa7b', 'request_id': 'req-c9051cfe-cdfb-4a5f-8598-7f829678183c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:33.446288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e18dbeb2f134edba56094a9ee6faa7b', 'request_id': 'req-c9051cfe-cdfb-4a5f-8598-7f829678183c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:33.464068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] The token cache is being invalidate because project 9e18dbeb2f134edba56094a9ee6faa7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:33.464602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:33.465738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:33.467435 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 979/1961] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:57:33.471009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 983/1962] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:33 2022] DELETE /identity/v3/projects/9e18dbeb2f134edba56094a9ee6faa7b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:57:33.482809 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76206cc-5ac9-4f20-afba-e06f06580ea7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:33.485128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d76206cc-5ac9-4f20-afba-e06f06580ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:33.592164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76206cc-5ac9-4f20-afba-e06f06580ea7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:33.671127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea41bd-bdda-4456-8e29-ed1a85673bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:33.671856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea41bd-bdda-4456-8e29-ed1a85673bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.672307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea41bd-bdda-4456-8e29-ed1a85673bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.672599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea41bd-bdda-4456-8e29-ed1a85673bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:33.674785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-56ea41bd-bdda-4456-8e29-ed1a85673bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:33.793352 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:33.796566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea41bd-bdda-4456-8e29-ed1a85673bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:33.797353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:33.799743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea41bd-bdda-4456-8e29-ed1a85673bb6 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:33.820500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9051cfe-cdfb-4a5f-8598-7f829678183c tempest-VolumeQuotaClassesTest-2042264537 tempest-VolumeQuotaClassesTest-2042264537-project-admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:33.821301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 980/1963] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:33.828321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:33.829402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.829780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.830182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] PATH_INFO: `/v3/users/d82dc0f11c0343e7ab6bd376a1d5800a` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:33.832192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d82dc0f11c0343e7ab6bd376a1d5800a)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:33.834514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:33.922214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd82dc0f11c0343e7ab6bd376a1d5800a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:33.944100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd82dc0f11c0343e7ab6bd376a1d5800a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:33.957506 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd82dc0f11c0343e7ab6bd376a1d5800a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:33.959105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd82dc0f11c0343e7ab6bd376a1d5800a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:33.971344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd82dc0f11c0343e7ab6bd376a1d5800a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:33.981722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 984/1964] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:33 2022] DELETE /identity/v3/users/d82dc0f11c0343e7ab6bd376a1d5800a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:33.990254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc726ea9-e8c4-4309-aca3-ad73e9eea0d2 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:33.990760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc726ea9-e8c4-4309-aca3-ad73e9eea0d2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:33.991233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc726ea9-e8c4-4309-aca3-ad73e9eea0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:34.022520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-bc726ea9-e8c4-4309-aca3-ad73e9eea0d2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:34.024842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-bc726ea9-e8c4-4309-aca3-ad73e9eea0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:34.143734 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 981/1965] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:57:34.156388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:34.158376 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:34.284988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faf94e-d2b0-44b4-acba-aee311911c6f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:34.291787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b187a-e69c-4fd6-bcc8-5b62ffc3507a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:34.292966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b187a-e69c-4fd6-bcc8-5b62ffc3507a service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:34.293500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b187a-e69c-4fd6-bcc8-5b62ffc3507a service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:34.293996 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b187a-e69c-4fd6-bcc8-5b62ffc3507a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:34.312979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b187a-e69c-4fd6-bcc8-5b62ffc3507a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:34.320094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b187a-e69c-4fd6-bcc8-5b62ffc3507a service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:34.340108 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 985/1966] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:35.251662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86fc3b-1077-4d20-8336-065da41aa3d7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:35.253395 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86fc3b-1077-4d20-8336-065da41aa3d7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:35.254194 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86fc3b-1077-4d20-8336-065da41aa3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:35.337044 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-8c86fc3b-1077-4d20-8336-065da41aa3d7 None None] MFA Rules not processed for user `902bea79f26a4c1d8c3c1937f99f9d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:35.339736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8c86fc3b-1077-4d20-8336-065da41aa3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:35.422756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b187a-e69c-4fd6-bcc8-5b62ffc3507a service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:35.425823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-527b187a-e69c-4fd6-bcc8-5b62ffc3507a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:35.452924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b187a-e69c-4fd6-bcc8-5b62ffc3507a service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:35.470110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:35.471266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:35.472114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:35.472420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] PATH_INFO: `/v3/projects/354352f72c3a49a6893e809c63a55d95` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:35.476033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] RBAC: Authorizing `identity:delete_project(project_id=354352f72c3a49a6893e809c63a55d95)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:35.479171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:35.501581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 982/1967] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:35.515605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86fc3b-1077-4d20-8336-065da41aa3d7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:35.520083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8c86fc3b-1077-4d20-8336-065da41aa3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:35.597424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354352f72c3a49a6893e809c63a55d95', 'request_id': 'req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:35.620299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354352f72c3a49a6893e809c63a55d95', 'request_id': 'req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:35.640271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354352f72c3a49a6893e809c63a55d95', 'request_id': 'req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:35.665948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] The token cache is being invalidate because project 354352f72c3a49a6893e809c63a55d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:35.667085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:35.669031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:35.672931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 986/1968] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:35 2022] DELETE /identity/v3/projects/354352f72c3a49a6893e809c63a55d95 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:57:35.687047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:35.690305 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:35.731213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86fc3b-1077-4d20-8336-065da41aa3d7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:35.839419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5516dc-2d4d-4a33-84c2-159b4c5f657d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:35.849954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a38d0-70c7-4783-965f-124b59ae1d6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:35.850845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a38d0-70c7-4783-965f-124b59ae1d6c service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:35.851684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:35.851918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a38d0-70c7-4783-965f-124b59ae1d6c service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:35.852384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a38d0-70c7-4783-965f-124b59ae1d6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:35.852765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:35.853100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:35.853577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] PATH_INFO: `/v3/users/d4cda3c1d52c4c61860da5e34aa3b1df` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:35.855412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4cda3c1d52c4c61860da5e34aa3b1df)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:35.855687 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-235a38d0-70c7-4783-965f-124b59ae1d6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:35.859600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:35.938245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4cda3c1d52c4c61860da5e34aa3b1df'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:35.956530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4cda3c1d52c4c61860da5e34aa3b1df'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:35.970868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a38d0-70c7-4783-965f-124b59ae1d6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:35.973016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a38d0-70c7-4783-965f-124b59ae1d6c service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:35.990599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4cda3c1d52c4c61860da5e34aa3b1df'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:35.992208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 983/1969] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:35.992387 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4cda3c1d52c4c61860da5e34aa3b1df'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:36.006430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-45063bc4-8abb-4e36-bd59-84879725fad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4cda3c1d52c4c61860da5e34aa3b1df'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:36.018359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 987/1970] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:35 2022] DELETE /identity/v3/users/d4cda3c1d52c4c61860da5e34aa3b1df => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:36.504516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a38d0-70c7-4783-965f-124b59ae1d6c service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:36.526862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a38d0-70c7-4783-965f-124b59ae1d6c service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:36.638379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:36.639960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:36.640471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:36.640942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] PATH_INFO: `/v3/projects/89c32586514b4e30b8cccde71fd9812e` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:36.643013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=89c32586514b4e30b8cccde71fd9812e)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:36.646923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:36.728126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c32586514b4e30b8cccde71fd9812e', 'request_id': 'req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:36.735087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c32586514b4e30b8cccde71fd9812e', 'request_id': 'req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:36.741046 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c32586514b4e30b8cccde71fd9812e', 'request_id': 'req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:36.759079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] The token cache is being invalidate because project 89c32586514b4e30b8cccde71fd9812e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:36.759551 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:36.760624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ae637c73-0a3e-442a-ae29-7ba3f7b250ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:36.763580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 984/1971] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:36 2022] DELETE /identity/v3/projects/89c32586514b4e30b8cccde71fd9812e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:36.770379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c0eea-e676-4d18-8310-bd4c8dbecd4e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:36.771031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c0eea-e676-4d18-8310-bd4c8dbecd4e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:36.771031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c0eea-e676-4d18-8310-bd4c8dbecd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:36.802451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-ed6c0eea-e676-4d18-8310-bd4c8dbecd4e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:36.803750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ed6c0eea-e676-4d18-8310-bd4c8dbecd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:36.916797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 988/1972] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:36.921000 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0f7bc-47e0-432e-9f7d-7d67b09708e6 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:36.921505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0f7bc-47e0-432e-9f7d-7d67b09708e6 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:36.921667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0f7bc-47e0-432e-9f7d-7d67b09708e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:36.944425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-94d0f7bc-47e0-432e-9f7d-7d67b09708e6 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:36.946318 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-94d0f7bc-47e0-432e-9f7d-7d67b09708e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:36.965055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 985/1973] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:36.965832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c0eea-e676-4d18-8310-bd4c8dbecd4e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:36.977201 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c0eea-e676-4d18-8310-bd4c8dbecd4e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:36.986311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54f384-0cc3-40c8-851c-193c47b99e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:36.987697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54f384-0cc3-40c8-851c-193c47b99e25 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:36.988174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54f384-0cc3-40c8-851c-193c47b99e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:36.988649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54f384-0cc3-40c8-851c-193c47b99e25 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:36.990171 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54f384-0cc3-40c8-851c-193c47b99e25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:36.993370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54f384-0cc3-40c8-851c-193c47b99e25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:36.994082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54f384-0cc3-40c8-851c-193c47b99e25 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:37.018774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 989/1974] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:57:37.021914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0f7bc-47e0-432e-9f7d-7d67b09708e6 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:37.039849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0f7bc-47e0-432e-9f7d-7d67b09708e6 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:37.048898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f247097-b2d7-4e8e-af8f-9b42e6c76825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:37.050908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f247097-b2d7-4e8e-af8f-9b42e6c76825 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:37.051124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f247097-b2d7-4e8e-af8f-9b42e6c76825 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:37.051695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f247097-b2d7-4e8e-af8f-9b42e6c76825 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:37.054586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f247097-b2d7-4e8e-af8f-9b42e6c76825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:37.060473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f247097-b2d7-4e8e-af8f-9b42e6c76825 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:37.138355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 986/1975] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:37 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:37.139066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54f384-0cc3-40c8-851c-193c47b99e25 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:37.142221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54f384-0cc3-40c8-851c-193c47b99e25 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:37.241060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2710-fdd1-46fe-8ba3-88a85a5a55ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:37.242202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2710-fdd1-46fe-8ba3-88a85a5a55ed admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:37.242752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2710-fdd1-46fe-8ba3-88a85a5a55ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:37.243301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2710-fdd1-46fe-8ba3-88a85a5a55ed admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:37.244532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a2710-fdd1-46fe-8ba3-88a85a5a55ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:37.249710 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a2710-fdd1-46fe-8ba3-88a85a5a55ed admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:37.286803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 990/1976] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:37 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:37.290154 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f247097-b2d7-4e8e-af8f-9b42e6c76825 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:37.314369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f247097-b2d7-4e8e-af8f-9b42e6c76825 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:37.322076 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd563cac-8122-44db-8ca9-bc22d1be1560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:37.323065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd563cac-8122-44db-8ca9-bc22d1be1560 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:37.324110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd563cac-8122-44db-8ca9-bc22d1be1560 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:37.324633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd563cac-8122-44db-8ca9-bc22d1be1560 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:37.326249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd563cac-8122-44db-8ca9-bc22d1be1560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:37.329184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd563cac-8122-44db-8ca9-bc22d1be1560 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:37.352208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 987/1977] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:57:37.364022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2710-fdd1-46fe-8ba3-88a85a5a55ed admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:37.382220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2710-fdd1-46fe-8ba3-88a85a5a55ed admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:37.390357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86c7c4-3941-43c2-8196-20ab562ac838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:37.391533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86c7c4-3941-43c2-8196-20ab562ac838 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:37.392024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86c7c4-3941-43c2-8196-20ab562ac838 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:37.392512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86c7c4-3941-43c2-8196-20ab562ac838 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:37.393850 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86c7c4-3941-43c2-8196-20ab562ac838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:37.396884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86c7c4-3941-43c2-8196-20ab562ac838 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:37.416275 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 991/1978] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:37.421726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd563cac-8122-44db-8ca9-bc22d1be1560 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:37.439372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd563cac-8122-44db-8ca9-bc22d1be1560 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:37.446662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e455e9a-bd78-4385-b1f6-3aea19ff1907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:37.447540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e455e9a-bd78-4385-b1f6-3aea19ff1907 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:37.447815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e455e9a-bd78-4385-b1f6-3aea19ff1907 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:37.448214 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e455e9a-bd78-4385-b1f6-3aea19ff1907 admin admin] PATH_INFO: `/v3/projects/dd6452bd0d354bcf843201865348aec6/users/3a4009c7b6154fc291a431f017ea5c13/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:37.472086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e455e9a-bd78-4385-b1f6-3aea19ff1907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6452bd0d354bcf843201865348aec6, user_id=3a4009c7b6154fc291a431f017ea5c13, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:37.475593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e455e9a-bd78-4385-b1f6-3aea19ff1907 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:37.496967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 988/1979] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:37 2022] PUT /identity/v3/projects/dd6452bd0d354bcf843201865348aec6/users/3a4009c7b6154fc291a431f017ea5c13/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:37.511940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571627ee-9f61-45ca-8aae-e98298c4fc36 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:37.512306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571627ee-9f61-45ca-8aae-e98298c4fc36 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:37.512495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571627ee-9f61-45ca-8aae-e98298c4fc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:37.568186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-571627ee-9f61-45ca-8aae-e98298c4fc36 None None] MFA Rules not processed for user `3a4009c7b6154fc291a431f017ea5c13`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:37.570602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-571627ee-9f61-45ca-8aae-e98298c4fc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:37.589156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cb3d6-7376-4744-9347-46de132b0540 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:37.590692 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cb3d6-7376-4744-9347-46de132b0540 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:37.591162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cb3d6-7376-4744-9347-46de132b0540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:37.614871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e89cb3d6-7376-4744-9347-46de132b0540 None None] MFA Rules not processed for user `902bea79f26a4c1d8c3c1937f99f9d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:37.619984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e89cb3d6-7376-4744-9347-46de132b0540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:37.675474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 992/1980] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:37.685743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571627ee-9f61-45ca-8aae-e98298c4fc36 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:37.746511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 989/1981] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:37.834751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571627ee-9f61-45ca-8aae-e98298c4fc36 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:37.872063 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97def0a3-c13e-4d4f-b7c5-566a2b0f0962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:37.872768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def0a3-c13e-4d4f-b7c5-566a2b0f0962 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:37.873050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def0a3-c13e-4d4f-b7c5-566a2b0f0962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:37.873323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def0a3-c13e-4d4f-b7c5-566a2b0f0962 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:37.874787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97def0a3-c13e-4d4f-b7c5-566a2b0f0962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:37.876745 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97def0a3-c13e-4d4f-b7c5-566a2b0f0962 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:37.922761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 993/1982] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:37 2022] POST /identity/v3/projects => generated 391 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:37.928909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89cb3d6-7376-4744-9347-46de132b0540 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:37.946160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89cb3d6-7376-4744-9347-46de132b0540 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:38.054078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47072d0-798b-437f-872b-ad6700e0fcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:38.054648 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47072d0-798b-437f-872b-ad6700e0fcb6 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:38.054969 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47072d0-798b-437f-872b-ad6700e0fcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:38.055252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47072d0-798b-437f-872b-ad6700e0fcb6 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:38.056227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47072d0-798b-437f-872b-ad6700e0fcb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:38.058211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47072d0-798b-437f-872b-ad6700e0fcb6 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:38.088767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 990/1983] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:37 2022] POST /identity/v3/users => generated 302 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:38.093870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97def0a3-c13e-4d4f-b7c5-566a2b0f0962 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:38.108355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97def0a3-c13e-4d4f-b7c5-566a2b0f0962 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:38.116427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf9c19-13da-4edc-8e79-9b12d37dc589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:38.117453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf9c19-13da-4edc-8e79-9b12d37dc589 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:38.118211 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf9c19-13da-4edc-8e79-9b12d37dc589 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:38.118211 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf9c19-13da-4edc-8e79-9b12d37dc589 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:38.118771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf9c19-13da-4edc-8e79-9b12d37dc589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:38.120504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf9c19-13da-4edc-8e79-9b12d37dc589 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:38.145498 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 994/1984] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:38.152092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47072d0-798b-437f-872b-ad6700e0fcb6 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:38.176833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47072d0-798b-437f-872b-ad6700e0fcb6 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:38.187826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20404b6c-7c87-4b00-bc32-159f77fdb61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:38.189433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20404b6c-7c87-4b00-bc32-159f77fdb61e admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:38.189915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20404b6c-7c87-4b00-bc32-159f77fdb61e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:38.190410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20404b6c-7c87-4b00-bc32-159f77fdb61e admin admin] PATH_INFO: `/v3/projects/112a8e60e33d40b1a1dc5656f42774bc/users/1cf166f7fb0e458f848d755334dbf5fe/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:38.220836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20404b6c-7c87-4b00-bc32-159f77fdb61e admin admin] RBAC: Authorizing `identity:create_grant(project_id=112a8e60e33d40b1a1dc5656f42774bc, user_id=1cf166f7fb0e458f848d755334dbf5fe, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:38.226379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20404b6c-7c87-4b00-bc32-159f77fdb61e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:38.253528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 991/1985] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:38 2022] PUT /identity/v3/projects/112a8e60e33d40b1a1dc5656f42774bc/users/1cf166f7fb0e458f848d755334dbf5fe/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:38.258698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf9c19-13da-4edc-8e79-9b12d37dc589 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:38.287978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf9c19-13da-4edc-8e79-9b12d37dc589 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:38.403736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe723a50-b8e8-49f2-9534-33788eb395dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:38.404594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723a50-b8e8-49f2-9534-33788eb395dc admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:38.404934 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723a50-b8e8-49f2-9534-33788eb395dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:38.405938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723a50-b8e8-49f2-9534-33788eb395dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:38.406851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe723a50-b8e8-49f2-9534-33788eb395dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:38.408515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe723a50-b8e8-49f2-9534-33788eb395dc admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:38.425078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 995/1986] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:57:38.437413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20404b6c-7c87-4b00-bc32-159f77fdb61e admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:38.449725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20404b6c-7c87-4b00-bc32-159f77fdb61e admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:38.456185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df9062-885f-43c9-8baa-5f5a66577c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:38.458226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df9062-885f-43c9-8baa-5f5a66577c5a admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:38.458983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df9062-885f-43c9-8baa-5f5a66577c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:38.458983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df9062-885f-43c9-8baa-5f5a66577c5a admin admin] PATH_INFO: `/v3/projects/112a8e60e33d40b1a1dc5656f42774bc/users/1cf166f7fb0e458f848d755334dbf5fe/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:38.461595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df9062-885f-43c9-8baa-5f5a66577c5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=112a8e60e33d40b1a1dc5656f42774bc, user_id=1cf166f7fb0e458f848d755334dbf5fe, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:38.466338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df9062-885f-43c9-8baa-5f5a66577c5a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:38.493647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 992/1987] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:38 2022] PUT /identity/v3/projects/112a8e60e33d40b1a1dc5656f42774bc/users/1cf166f7fb0e458f848d755334dbf5fe/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:38.502045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44672a8d-d0cc-4676-be2f-75b439d16a8d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:38.502557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44672a8d-d0cc-4676-be2f-75b439d16a8d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:38.503096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44672a8d-d0cc-4676-be2f-75b439d16a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:38.546051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-44672a8d-d0cc-4676-be2f-75b439d16a8d None None] MFA Rules not processed for user `1cf166f7fb0e458f848d755334dbf5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:38.548394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-44672a8d-d0cc-4676-be2f-75b439d16a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:38.681115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 996/1988] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:38.698660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df9062-885f-43c9-8baa-5f5a66577c5a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:38.701567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-51df9062-885f-43c9-8baa-5f5a66577c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:38.800902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df9062-885f-43c9-8baa-5f5a66577c5a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:38.813256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bb448-820b-4740-ac06-8c8a51fed0c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:38.814406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bb448-820b-4740-ac06-8c8a51fed0c5 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:38.814962 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bb448-820b-4740-ac06-8c8a51fed0c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:38.816038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bb448-820b-4740-ac06-8c8a51fed0c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:38.838055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bb448-820b-4740-ac06-8c8a51fed0c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:38.841472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bb448-820b-4740-ac06-8c8a51fed0c5 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:38.858335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 993/1989] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:38.951807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44672a8d-d0cc-4676-be2f-75b439d16a8d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:38.960951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-44672a8d-d0cc-4676-be2f-75b439d16a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:38.988897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44672a8d-d0cc-4676-be2f-75b439d16a8d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:38.999517 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab53f07-a346-4de5-bc54-f5204aee0d4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:39.001711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53f07-a346-4de5-bc54-f5204aee0d4b service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:39.002924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53f07-a346-4de5-bc54-f5204aee0d4b service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:39.003537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53f07-a346-4de5-bc54-f5204aee0d4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:39.100585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab53f07-a346-4de5-bc54-f5204aee0d4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:39.102738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab53f07-a346-4de5-bc54-f5204aee0d4b service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:39.116845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 997/1990] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:39.633106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bb448-820b-4740-ac06-8c8a51fed0c5 service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:39.635585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4b2bb448-820b-4740-ac06-8c8a51fed0c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:39.760114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bb448-820b-4740-ac06-8c8a51fed0c5 service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:39.766072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:39.767221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:39.767371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:39.767511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] PATH_INFO: `/v3/users/dc2aff3e556549e2b6f0de51623674b4` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:39.768755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2aff3e556549e2b6f0de51623674b4)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:39.770696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:39.834443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2aff3e556549e2b6f0de51623674b4'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:39.835161 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2aff3e556549e2b6f0de51623674b4'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:39.854469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2aff3e556549e2b6f0de51623674b4'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:39.856184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2aff3e556549e2b6f0de51623674b4'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:39.868991 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2aff3e556549e2b6f0de51623674b4'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:39.888819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 994/1991] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:39 2022] DELETE /identity/v3/users/dc2aff3e556549e2b6f0de51623674b4 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:39.894534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258ad6-096b-4977-b568-61a236539585 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:39.895016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258ad6-096b-4977-b568-61a236539585 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:39.895303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258ad6-096b-4977-b568-61a236539585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:39.913678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-86258ad6-096b-4977-b568-61a236539585 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:39.914960 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-86258ad6-096b-4977-b568-61a236539585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:40.003083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 998/1992] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:57:40.027754 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:40.029776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:40.104206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fd0cf-a529-47cf-a98c-cd70a3814e33 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:40.106386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e1abf-63e1-49b9-85b1-248b771fcf2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:40.107117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e1abf-63e1-49b9-85b1-248b771fcf2d service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:40.107401 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e1abf-63e1-49b9-85b1-248b771fcf2d service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:40.107676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e1abf-63e1-49b9-85b1-248b771fcf2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:40.119773 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e1abf-63e1-49b9-85b1-248b771fcf2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:40.122172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e1abf-63e1-49b9-85b1-248b771fcf2d service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:40.136865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 995/1993] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:40.442017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258ad6-096b-4977-b568-61a236539585 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:40.444121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-86258ad6-096b-4977-b568-61a236539585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:40.463503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258ad6-096b-4977-b568-61a236539585 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:40.478796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:40.480004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:40.480531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:40.480955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] PATH_INFO: `/v3/projects/9e16f3e5dc0046fdb80c08e3d4853b24` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:40.483766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e16f3e5dc0046fdb80c08e3d4853b24)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:40.486743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:40.553374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e16f3e5dc0046fdb80c08e3d4853b24', 'request_id': 'req-176abd1d-9632-46b0-bd93-94030c5ea671'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:40.563380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e16f3e5dc0046fdb80c08e3d4853b24', 'request_id': 'req-176abd1d-9632-46b0-bd93-94030c5ea671'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:40.573879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e16f3e5dc0046fdb80c08e3d4853b24', 'request_id': 'req-176abd1d-9632-46b0-bd93-94030c5ea671'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:40.589382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] The token cache is being invalidate because project 9e16f3e5dc0046fdb80c08e3d4853b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:40.589732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:40.591083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-176abd1d-9632-46b0-bd93-94030c5ea671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:40.592938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 999/1994] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:40 2022] DELETE /identity/v3/projects/9e16f3e5dc0046fdb80c08e3d4853b24 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:40.597921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549ef5a-1694-4a75-9e2d-68260e171134 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:40.598204 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549ef5a-1694-4a75-9e2d-68260e171134 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:40.598477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549ef5a-1694-4a75-9e2d-68260e171134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:40.620664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-5549ef5a-1694-4a75-9e2d-68260e171134 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:40.621970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5549ef5a-1694-4a75-9e2d-68260e171134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:40.708516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 996/1995] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:40.720770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96d81f-5ef8-47d5-98b9-a3683ad05918 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:40.721244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96d81f-5ef8-47d5-98b9-a3683ad05918 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:40.722009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96d81f-5ef8-47d5-98b9-a3683ad05918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:40.747530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-3a96d81f-5ef8-47d5-98b9-a3683ad05918 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:40.748659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-3a96d81f-5ef8-47d5-98b9-a3683ad05918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:40.758275 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1000/1996] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:40.764259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549ef5a-1694-4a75-9e2d-68260e171134 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:40.780118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549ef5a-1694-4a75-9e2d-68260e171134 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:40.787217 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b92bc-3e4b-4da5-b7cc-378c102f6cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:40.787998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b92bc-3e4b-4da5-b7cc-378c102f6cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:40.788469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b92bc-3e4b-4da5-b7cc-378c102f6cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:40.788818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b92bc-3e4b-4da5-b7cc-378c102f6cd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:40.789681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b92bc-3e4b-4da5-b7cc-378c102f6cd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:40.790171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b92bc-3e4b-4da5-b7cc-378c102f6cd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:40.792237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b92bc-3e4b-4da5-b7cc-378c102f6cd1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:40.821327 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 997/1997] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:57:40.829678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96d81f-5ef8-47d5-98b9-a3683ad05918 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:40.845983 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96d81f-5ef8-47d5-98b9-a3683ad05918 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:40.852759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae00c4-dbd5-484f-8f42-75da7f4529c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:40.854240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae00c4-dbd5-484f-8f42-75da7f4529c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:40.854693 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae00c4-dbd5-484f-8f42-75da7f4529c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:40.855191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae00c4-dbd5-484f-8f42-75da7f4529c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:40.856462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae00c4-dbd5-484f-8f42-75da7f4529c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:40.861193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae00c4-dbd5-484f-8f42-75da7f4529c9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:40.922335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1001/1998] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:40 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:40.929973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b92bc-3e4b-4da5-b7cc-378c102f6cd1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:40.945905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b92bc-3e4b-4da5-b7cc-378c102f6cd1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:41.053088 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ff5a8-caab-4603-88d8-469170f3a6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:41.053902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ff5a8-caab-4603-88d8-469170f3a6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.054254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ff5a8-caab-4603-88d8-469170f3a6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.054590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ff5a8-caab-4603-88d8-469170f3a6e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.055656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ff5a8-caab-4603-88d8-469170f3a6e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:41.057885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ff5a8-caab-4603-88d8-469170f3a6e2 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:41.083442 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 998/1999] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:40 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:41.087754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae00c4-dbd5-484f-8f42-75da7f4529c9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:41.100115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae00c4-dbd5-484f-8f42-75da7f4529c9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:41.108674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327cf28-4fd4-4704-bd32-3498652a8435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:41.109628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327cf28-4fd4-4704-bd32-3498652a8435 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.110174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327cf28-4fd4-4704-bd32-3498652a8435 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.110991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327cf28-4fd4-4704-bd32-3498652a8435 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.113533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327cf28-4fd4-4704-bd32-3498652a8435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:41.115894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327cf28-4fd4-4704-bd32-3498652a8435 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:41.133368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1002/2000] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:41.137805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ff5a8-caab-4603-88d8-469170f3a6e2 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:41.149029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ff5a8-caab-4603-88d8-469170f3a6e2 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:41.156930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0cb56-4aae-4c1e-a472-9b08d8636ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:41.158086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0cb56-4aae-4c1e-a472-9b08d8636ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.158864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0cb56-4aae-4c1e-a472-9b08d8636ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.162994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0cb56-4aae-4c1e-a472-9b08d8636ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.164812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0cb56-4aae-4c1e-a472-9b08d8636ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:41.168656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0cb56-4aae-4c1e-a472-9b08d8636ad7 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:41.184471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 999/2001] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:41.190999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327cf28-4fd4-4704-bd32-3498652a8435 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:41.207595 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327cf28-4fd4-4704-bd32-3498652a8435 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:41.214073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488ce13-28fa-41d6-9a14-03e3eb1529f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:41.216121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488ce13-28fa-41d6-9a14-03e3eb1529f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.216580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488ce13-28fa-41d6-9a14-03e3eb1529f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.217051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488ce13-28fa-41d6-9a14-03e3eb1529f3 admin admin] PATH_INFO: `/v3/projects/976b5c9e03624e1797866400092aa757/users/04fdd1810ec24077a7dc4d34759bc801/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.239018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488ce13-28fa-41d6-9a14-03e3eb1529f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=976b5c9e03624e1797866400092aa757, user_id=04fdd1810ec24077a7dc4d34759bc801, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:41.242859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488ce13-28fa-41d6-9a14-03e3eb1529f3 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:41.262441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1003/2002] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:41 2022] PUT /identity/v3/projects/976b5c9e03624e1797866400092aa757/users/04fdd1810ec24077a7dc4d34759bc801/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:41.279233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75ebb3-10ee-40d5-91c0-86d36fcb9c7f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.279519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75ebb3-10ee-40d5-91c0-86d36fcb9c7f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.279806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75ebb3-10ee-40d5-91c0-86d36fcb9c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.311775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-5c75ebb3-10ee-40d5-91c0-86d36fcb9c7f None None] MFA Rules not processed for user `04fdd1810ec24077a7dc4d34759bc801`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:41.313991 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5c75ebb3-10ee-40d5-91c0-86d36fcb9c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:41.416980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1000/2003] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:41.421200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488ce13-28fa-41d6-9a14-03e3eb1529f3 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:41.505198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488ce13-28fa-41d6-9a14-03e3eb1529f3 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:41.510141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91abb8-5a01-4138-89d1-7a68d991bebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:41.510846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91abb8-5a01-4138-89d1-7a68d991bebc admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.511124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91abb8-5a01-4138-89d1-7a68d991bebc admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.511424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91abb8-5a01-4138-89d1-7a68d991bebc admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.512182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91abb8-5a01-4138-89d1-7a68d991bebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:41.514206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91abb8-5a01-4138-89d1-7a68d991bebc admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:41.578938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1004/2004] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:41 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:57:41.581640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75ebb3-10ee-40d5-91c0-86d36fcb9c7f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:41.610787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75ebb3-10ee-40d5-91c0-86d36fcb9c7f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:41.724908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db5a09-5f49-47b6-9597-f121159977ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:41.725690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db5a09-5f49-47b6-9597-f121159977ca admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.726351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db5a09-5f49-47b6-9597-f121159977ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.726702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db5a09-5f49-47b6-9597-f121159977ca admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.727546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db5a09-5f49-47b6-9597-f121159977ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:41.729556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db5a09-5f49-47b6-9597-f121159977ca admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:41.751470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1001/2005] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:41 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:57:41.760294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91abb8-5a01-4138-89d1-7a68d991bebc admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:41.775106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91abb8-5a01-4138-89d1-7a68d991bebc admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:41.780588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453de97-fae5-47dc-bb99-0ad83997b1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:41.781440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453de97-fae5-47dc-bb99-0ad83997b1fd admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.781735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453de97-fae5-47dc-bb99-0ad83997b1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.782009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453de97-fae5-47dc-bb99-0ad83997b1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.782799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453de97-fae5-47dc-bb99-0ad83997b1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:41.784623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453de97-fae5-47dc-bb99-0ad83997b1fd admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:41.801520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1005/2006] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:41.802865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db5a09-5f49-47b6-9597-f121159977ca admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:41.814210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db5a09-5f49-47b6-9597-f121159977ca admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:41.819309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7f063-fd89-4afb-9add-f1d7ac04b930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:41.820242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7f063-fd89-4afb-9add-f1d7ac04b930 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.820550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7f063-fd89-4afb-9add-f1d7ac04b930 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.820821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7f063-fd89-4afb-9add-f1d7ac04b930 admin admin] PATH_INFO: `/v3/projects/eb9722dd4eb044ba89c53081ba4104eb/users/f3068a154e0d4c1dbeed3b8b8840793b/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.837774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7f063-fd89-4afb-9add-f1d7ac04b930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9722dd4eb044ba89c53081ba4104eb, user_id=f3068a154e0d4c1dbeed3b8b8840793b, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:41.841065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7f063-fd89-4afb-9add-f1d7ac04b930 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:41.857893 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1002/2007] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:41 2022] PUT /identity/v3/projects/eb9722dd4eb044ba89c53081ba4104eb/users/f3068a154e0d4c1dbeed3b8b8840793b/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:41.869705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453de97-fae5-47dc-bb99-0ad83997b1fd admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:41.874811 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453de97-fae5-47dc-bb99-0ad83997b1fd admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:41.940349 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6c7e7-a12b-4824-a0a4-41fafad46a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:41.941032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6c7e7-a12b-4824-a0a4-41fafad46a18 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.941315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6c7e7-a12b-4824-a0a4-41fafad46a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.941590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6c7e7-a12b-4824-a0a4-41fafad46a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.942324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6c7e7-a12b-4824-a0a4-41fafad46a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:41.943977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6c7e7-a12b-4824-a0a4-41fafad46a18 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:41.955112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1006/2008] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:57:41.961215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7f063-fd89-4afb-9add-f1d7ac04b930 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:41.971636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7f063-fd89-4afb-9add-f1d7ac04b930 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:41.976918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46c3bc-3e94-4d3a-bea6-f8c6df84ead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:41.977898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46c3bc-3e94-4d3a-bea6-f8c6df84ead9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:41.978197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46c3bc-3e94-4d3a-bea6-f8c6df84ead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:41.978464 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46c3bc-3e94-4d3a-bea6-f8c6df84ead9 admin admin] PATH_INFO: `/v3/projects/eb9722dd4eb044ba89c53081ba4104eb/users/f3068a154e0d4c1dbeed3b8b8840793b/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:41.981142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46c3bc-3e94-4d3a-bea6-f8c6df84ead9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9722dd4eb044ba89c53081ba4104eb, user_id=f3068a154e0d4c1dbeed3b8b8840793b, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:41.984101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46c3bc-3e94-4d3a-bea6-f8c6df84ead9 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:41.995859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1003/2009] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:41 2022] PUT /identity/v3/projects/eb9722dd4eb044ba89c53081ba4104eb/users/f3068a154e0d4c1dbeed3b8b8840793b/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:42.009605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3469-7c99-483d-8cad-a64efeb6593e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:42.009901 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3469-7c99-483d-8cad-a64efeb6593e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:42.010176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3469-7c99-483d-8cad-a64efeb6593e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:42.043981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1e4c3469-7c99-483d-8cad-a64efeb6593e None None] MFA Rules not processed for user `f3068a154e0d4c1dbeed3b8b8840793b`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:42.045907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1e4c3469-7c99-483d-8cad-a64efeb6593e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:42.162255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1007/2010] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:42.179832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46c3bc-3e94-4d3a-bea6-f8c6df84ead9 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:42.184159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4f46c3bc-3e94-4d3a-bea6-f8c6df84ead9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:42.321876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46c3bc-3e94-4d3a-bea6-f8c6df84ead9 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:42.326987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac71e9-772f-4360-b6a2-914963ebc277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:42.334942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac71e9-772f-4360-b6a2-914963ebc277 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:42.334942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac71e9-772f-4360-b6a2-914963ebc277 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:42.334942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac71e9-772f-4360-b6a2-914963ebc277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:42.422502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac71e9-772f-4360-b6a2-914963ebc277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:42.424386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac71e9-772f-4360-b6a2-914963ebc277 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:42.435264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1004/2011] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:43.100549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c3469-7c99-483d-8cad-a64efeb6593e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:43.102607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1e4c3469-7c99-483d-8cad-a64efeb6593e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:43.205847 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c3469-7c99-483d-8cad-a64efeb6593e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:43.211225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:43.212613 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:43.213067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:43.213522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] PATH_INFO: `/v3/users/3a4009c7b6154fc291a431f017ea5c13` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:43.216294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a4009c7b6154fc291a431f017ea5c13)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:43.219350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:43.290984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a4009c7b6154fc291a431f017ea5c13'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:43.312918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a4009c7b6154fc291a431f017ea5c13'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:43.328744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a4009c7b6154fc291a431f017ea5c13'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:43.329598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a4009c7b6154fc291a431f017ea5c13'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:43.342829 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a4009c7b6154fc291a431f017ea5c13'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:43.361191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1008/2012] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:43 2022] DELETE /identity/v3/users/3a4009c7b6154fc291a431f017ea5c13 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:43.363977 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec97b4-5bc1-4ef3-80ff-7c67b7b0b1e2 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:43.364610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec97b4-5bc1-4ef3-80ff-7c67b7b0b1e2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:43.365439 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec97b4-5bc1-4ef3-80ff-7c67b7b0b1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:43.390608 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-85ec97b4-5bc1-4ef3-80ff-7c67b7b0b1e2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:43.392445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-85ec97b4-5bc1-4ef3-80ff-7c67b7b0b1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:43.481254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1005/2013] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:43.494107 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:43.497649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:43.568342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e1495-700c-4345-9b48-bcbea0014dc8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:43.572235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ab8e7-920b-4157-8c63-5356a45f80c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:43.572896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ab8e7-920b-4157-8c63-5356a45f80c1 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:43.573159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ab8e7-920b-4157-8c63-5356a45f80c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:43.573432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ab8e7-920b-4157-8c63-5356a45f80c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:43.585534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ab8e7-920b-4157-8c63-5356a45f80c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:43.587400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ab8e7-920b-4157-8c63-5356a45f80c1 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:43.599412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1009/2014] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:43.905768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec97b4-5bc1-4ef3-80ff-7c67b7b0b1e2 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:43.907267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-85ec97b4-5bc1-4ef3-80ff-7c67b7b0b1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:43.920419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec97b4-5bc1-4ef3-80ff-7c67b7b0b1e2 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:43.925295 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:43.926314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:43.926597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:43.926897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] PATH_INFO: `/v3/projects/dd6452bd0d354bcf843201865348aec6` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:43.928138 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6452bd0d354bcf843201865348aec6)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:43.930066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:43.984543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6452bd0d354bcf843201865348aec6', 'request_id': 'req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:43.992561 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6452bd0d354bcf843201865348aec6', 'request_id': 'req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.001677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6452bd0d354bcf843201865348aec6', 'request_id': 'req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.022096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] The token cache is being invalidate because project dd6452bd0d354bcf843201865348aec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:44.022455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.023253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.024868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1006/2015] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:43 2022] DELETE /identity/v3/projects/dd6452bd0d354bcf843201865348aec6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:44.029470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ab8e7-920b-4157-8c63-5356a45f80c1 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:44.049124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ab8e7-920b-4157-8c63-5356a45f80c1 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:44.120361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:44.122338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:44.122628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:44.122964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] PATH_INFO: `/v3/users/1cf166f7fb0e458f848d755334dbf5fe` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:44.124685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cf166f7fb0e458f848d755334dbf5fe)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:44.127840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:44.196110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cf166f7fb0e458f848d755334dbf5fe'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.210378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cf166f7fb0e458f848d755334dbf5fe'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.231827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cf166f7fb0e458f848d755334dbf5fe'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.232999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cf166f7fb0e458f848d755334dbf5fe'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.257109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-92f85253-3e09-4607-a13c-3ae66886613e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cf166f7fb0e458f848d755334dbf5fe'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.266997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1010/2016] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:44 2022] DELETE /identity/v3/users/1cf166f7fb0e458f848d755334dbf5fe => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:57:44.628083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:44.630746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:44.646268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45de3c-ed89-4e07-88a9-5863f5ae33e5 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:44.749853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:44.750920 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:44.751192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:44.751565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] PATH_INFO: `/v3/projects/112a8e60e33d40b1a1dc5656f42774bc` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:44.753291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=112a8e60e33d40b1a1dc5656f42774bc)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:44.755171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:44.814032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '112a8e60e33d40b1a1dc5656f42774bc', 'request_id': 'req-7347c845-d902-4494-a057-421cd0f8eed8'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.820968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '112a8e60e33d40b1a1dc5656f42774bc', 'request_id': 'req-7347c845-d902-4494-a057-421cd0f8eed8'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.826673 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '112a8e60e33d40b1a1dc5656f42774bc', 'request_id': 'req-7347c845-d902-4494-a057-421cd0f8eed8'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.839391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] The token cache is being invalidate because project 112a8e60e33d40b1a1dc5656f42774bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:44.840211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.840857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7347c845-d902-4494-a057-421cd0f8eed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:44.842398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1007/2017] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:44 2022] DELETE /identity/v3/projects/112a8e60e33d40b1a1dc5656f42774bc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:44.847578 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c4cba-0793-442b-b380-2f5c035266cb None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:44.847964 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c4cba-0793-442b-b380-2f5c035266cb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:44.848304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c4cba-0793-442b-b380-2f5c035266cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:44.867107 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-437c4cba-0793-442b-b380-2f5c035266cb None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:44.868426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-437c4cba-0793-442b-b380-2f5c035266cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:44.937281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1011/2018] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:57:44.945368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94251f-68b7-4074-bd1c-a36b5f5a74a3 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:44.945658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94251f-68b7-4074-bd1c-a36b5f5a74a3 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:44.945935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94251f-68b7-4074-bd1c-a36b5f5a74a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:44.971100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-dc94251f-68b7-4074-bd1c-a36b5f5a74a3 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:44.972481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-dc94251f-68b7-4074-bd1c-a36b5f5a74a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:44.981408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1008/2019] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:44.985861 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c4cba-0793-442b-b380-2f5c035266cb None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:44.996035 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c4cba-0793-442b-b380-2f5c035266cb None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.000866 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b089ef8-e6b8-4e9d-87b5-c81f1736cdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.001524 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b089ef8-e6b8-4e9d-87b5-c81f1736cdeb admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.001820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b089ef8-e6b8-4e9d-87b5-c81f1736cdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.002072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b089ef8-e6b8-4e9d-87b5-c81f1736cdeb admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.002851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b089ef8-e6b8-4e9d-87b5-c81f1736cdeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:45.003152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b089ef8-e6b8-4e9d-87b5-c81f1736cdeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.004746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b089ef8-e6b8-4e9d-87b5-c81f1736cdeb admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.016513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1012/2020] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:57:45.020818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94251f-68b7-4074-bd1c-a36b5f5a74a3 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.030489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94251f-68b7-4074-bd1c-a36b5f5a74a3 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.035178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccbcf4-fbbf-49a7-9538-9780aef6175d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.035826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccbcf4-fbbf-49a7-9538-9780aef6175d admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.036108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccbcf4-fbbf-49a7-9538-9780aef6175d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.036381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccbcf4-fbbf-49a7-9538-9780aef6175d admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.037068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccbcf4-fbbf-49a7-9538-9780aef6175d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.039003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccbcf4-fbbf-49a7-9538-9780aef6175d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.066040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1009/2021] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:45 2022] POST /identity/v3/projects => generated 407 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:45.070333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b089ef8-e6b8-4e9d-87b5-c81f1736cdeb admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.080492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b089ef8-e6b8-4e9d-87b5-c81f1736cdeb admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.137512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bbcef-8b10-43bb-9162-4bf5c30cc78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.138183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bbcef-8b10-43bb-9162-4bf5c30cc78f admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.138462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bbcef-8b10-43bb-9162-4bf5c30cc78f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.138759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bbcef-8b10-43bb-9162-4bf5c30cc78f admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.139447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544bbcef-8b10-43bb-9162-4bf5c30cc78f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.141360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544bbcef-8b10-43bb-9162-4bf5c30cc78f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.162057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1013/2022] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:45 2022] POST /identity/v3/users => generated 304 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:45.166228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccbcf4-fbbf-49a7-9538-9780aef6175d admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.178793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccbcf4-fbbf-49a7-9538-9780aef6175d admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.183733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f39715-a846-4aba-8ab4-f6f4989afdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.184838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f39715-a846-4aba-8ab4-f6f4989afdcb admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.185122 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f39715-a846-4aba-8ab4-f6f4989afdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.185400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f39715-a846-4aba-8ab4-f6f4989afdcb admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.186231 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f39715-a846-4aba-8ab4-f6f4989afdcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.188051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f39715-a846-4aba-8ab4-f6f4989afdcb admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.200034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1010/2023] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:45.204602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bbcef-8b10-43bb-9162-4bf5c30cc78f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.214460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bbcef-8b10-43bb-9162-4bf5c30cc78f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.220365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352d598-ef24-4c98-8ab4-f3c6e7c194f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.221162 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352d598-ef24-4c98-8ab4-f3c6e7c194f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.221463 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352d598-ef24-4c98-8ab4-f3c6e7c194f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.221738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352d598-ef24-4c98-8ab4-f3c6e7c194f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.222458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352d598-ef24-4c98-8ab4-f3c6e7c194f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.224104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352d598-ef24-4c98-8ab4-f3c6e7c194f2 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.234349 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1014/2024] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:45.239449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f39715-a846-4aba-8ab4-f6f4989afdcb admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.251841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f39715-a846-4aba-8ab4-f6f4989afdcb admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.256553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc311d-1d49-4882-8c23-ee247c1bc4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.257433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc311d-1d49-4882-8c23-ee247c1bc4ae admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.257715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc311d-1d49-4882-8c23-ee247c1bc4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.257990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc311d-1d49-4882-8c23-ee247c1bc4ae admin admin] PATH_INFO: `/v3/projects/6840ff440be646858d4cfb6ec3fe9760/users/2eee74ae57f44c84bbaa7f085dce5520/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.274484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc311d-1d49-4882-8c23-ee247c1bc4ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6840ff440be646858d4cfb6ec3fe9760, user_id=2eee74ae57f44c84bbaa7f085dce5520, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.277819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc311d-1d49-4882-8c23-ee247c1bc4ae admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.290524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1011/2025] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:45 2022] PUT /identity/v3/projects/6840ff440be646858d4cfb6ec3fe9760/users/2eee74ae57f44c84bbaa7f085dce5520/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:45.301123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6231d01-55c4-4488-a8c9-62a6fec031b8 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.301438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6231d01-55c4-4488-a8c9-62a6fec031b8 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.301713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6231d01-55c4-4488-a8c9-62a6fec031b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.330234 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-c6231d01-55c4-4488-a8c9-62a6fec031b8 None None] MFA Rules not processed for user `2eee74ae57f44c84bbaa7f085dce5520`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:45.331481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c6231d01-55c4-4488-a8c9-62a6fec031b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:45.396449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1015/2026] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:45 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:45.401038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc311d-1d49-4882-8c23-ee247c1bc4ae admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.464615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc311d-1d49-4882-8c23-ee247c1bc4ae admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.469979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349645f4-817f-44cd-beec-b71cc14ffecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.470645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349645f4-817f-44cd-beec-b71cc14ffecf admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.470954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349645f4-817f-44cd-beec-b71cc14ffecf admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.471224 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349645f4-817f-44cd-beec-b71cc14ffecf admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.471961 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349645f4-817f-44cd-beec-b71cc14ffecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.473933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349645f4-817f-44cd-beec-b71cc14ffecf admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.507238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1012/2027] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:45 2022] POST /identity/v3/projects => generated 407 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:45.512126 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6231d01-55c4-4488-a8c9-62a6fec031b8 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.522817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6231d01-55c4-4488-a8c9-62a6fec031b8 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.603808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a5b2b-d8e2-4722-a408-96a77c31c64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.604544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5b2b-d8e2-4722-a408-96a77c31c64f admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.604834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5b2b-d8e2-4722-a408-96a77c31c64f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.605158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5b2b-d8e2-4722-a408-96a77c31c64f admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.605863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a5b2b-d8e2-4722-a408-96a77c31c64f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.607949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a5b2b-d8e2-4722-a408-96a77c31c64f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.630110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1016/2028] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:45 2022] POST /identity/v3/users => generated 310 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:57:45.635033 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349645f4-817f-44cd-beec-b71cc14ffecf admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.646872 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349645f4-817f-44cd-beec-b71cc14ffecf admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.651543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd6dd6-b279-4639-b503-f38119354002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.652233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6dd6-b279-4639-b503-f38119354002 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.652511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6dd6-b279-4639-b503-f38119354002 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.652807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6dd6-b279-4639-b503-f38119354002 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.653500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd6dd6-b279-4639-b503-f38119354002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.655149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd6dd6-b279-4639-b503-f38119354002 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.665954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1013/2029] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:45.670348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a5b2b-d8e2-4722-a408-96a77c31c64f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.680690 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a5b2b-d8e2-4722-a408-96a77c31c64f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.686102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77302bbd-1619-4443-a680-cdca06d179db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.687021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77302bbd-1619-4443-a680-cdca06d179db admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.687325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77302bbd-1619-4443-a680-cdca06d179db admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.687587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77302bbd-1619-4443-a680-cdca06d179db admin admin] PATH_INFO: `/v3/projects/19df30d3301a4338b5c551e7fc40c251/users/fe8cb9e8af0e4b5c9612a63981bfee47/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.706855 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77302bbd-1619-4443-a680-cdca06d179db admin admin] RBAC: Authorizing `identity:create_grant(project_id=19df30d3301a4338b5c551e7fc40c251, user_id=fe8cb9e8af0e4b5c9612a63981bfee47, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.709483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77302bbd-1619-4443-a680-cdca06d179db admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.722656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1017/2030] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:45 2022] PUT /identity/v3/projects/19df30d3301a4338b5c551e7fc40c251/users/fe8cb9e8af0e4b5c9612a63981bfee47/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:57:45.727121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd6dd6-b279-4639-b503-f38119354002 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.737860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd6dd6-b279-4639-b503-f38119354002 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.799410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634e15a-66a7-4d74-84ba-1d7ddfea3d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.800084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634e15a-66a7-4d74-84ba-1d7ddfea3d22 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.800359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634e15a-66a7-4d74-84ba-1d7ddfea3d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.800626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634e15a-66a7-4d74-84ba-1d7ddfea3d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.801341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634e15a-66a7-4d74-84ba-1d7ddfea3d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.802974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634e15a-66a7-4d74-84ba-1d7ddfea3d22 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.813807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1014/2031] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:57:45.818539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77302bbd-1619-4443-a680-cdca06d179db admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.830168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77302bbd-1619-4443-a680-cdca06d179db admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:45.835561 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242cf2c-6ce7-46ce-9643-1759cd0ad79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:45.836484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242cf2c-6ce7-46ce-9643-1759cd0ad79a admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.836726 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242cf2c-6ce7-46ce-9643-1759cd0ad79a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.837001 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242cf2c-6ce7-46ce-9643-1759cd0ad79a admin admin] PATH_INFO: `/v3/projects/19df30d3301a4338b5c551e7fc40c251/users/fe8cb9e8af0e4b5c9612a63981bfee47/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.838999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242cf2c-6ce7-46ce-9643-1759cd0ad79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19df30d3301a4338b5c551e7fc40c251, user_id=fe8cb9e8af0e4b5c9612a63981bfee47, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:45.841629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242cf2c-6ce7-46ce-9643-1759cd0ad79a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:45.852621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1018/2032] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:45 2022] PUT /identity/v3/projects/19df30d3301a4338b5c551e7fc40c251/users/fe8cb9e8af0e4b5c9612a63981bfee47/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:57:45.864677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0088f0-6bfb-436b-8b63-01c8aafc1bbb None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:45.864962 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0088f0-6bfb-436b-8b63-01c8aafc1bbb None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:45.865240 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0088f0-6bfb-436b-8b63-01c8aafc1bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:45.894735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-9f0088f0-6bfb-436b-8b63-01c8aafc1bbb None None] MFA Rules not processed for user `fe8cb9e8af0e4b5c9612a63981bfee47`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:45.896016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9f0088f0-6bfb-436b-8b63-01c8aafc1bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:45.978408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1015/2033] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:45 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:45.997297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242cf2c-6ce7-46ce-9643-1759cd0ad79a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:45.999706 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c242cf2c-6ce7-46ce-9643-1759cd0ad79a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:46.069457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242cf2c-6ce7-46ce-9643-1759cd0ad79a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:46.073697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa1a7f-ead7-4dd4-a7bb-a12ade39ded3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:46.074359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1a7f-ead7-4dd4-a7bb-a12ade39ded3 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:46.074687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1a7f-ead7-4dd4-a7bb-a12ade39ded3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:46.074941 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa1a7f-ead7-4dd4-a7bb-a12ade39ded3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:46.085540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa1a7f-ead7-4dd4-a7bb-a12ade39ded3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:46.087051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa1a7f-ead7-4dd4-a7bb-a12ade39ded3 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:46.097480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1019/2034] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:45 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:46.273522 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0088f0-6bfb-436b-8b63-01c8aafc1bbb None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:46.284735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0088f0-6bfb-436b-8b63-01c8aafc1bbb None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:46.288464 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fa05a-2dc8-4346-bc84-f94ac5e67bac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:46.289156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa05a-2dc8-4346-bc84-f94ac5e67bac service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:46.289531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa05a-2dc8-4346-bc84-f94ac5e67bac service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:46.289806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa05a-2dc8-4346-bc84-f94ac5e67bac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:46.356444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fa05a-2dc8-4346-bc84-f94ac5e67bac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:46.357934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fa05a-2dc8-4346-bc84-f94ac5e67bac service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:46.367479 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1016/2035] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:46 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:48.483021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa1a7f-ead7-4dd4-a7bb-a12ade39ded3 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:48.485469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9baa1a7f-ead7-4dd4-a7bb-a12ade39ded3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:48.514562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa1a7f-ead7-4dd4-a7bb-a12ade39ded3 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:48.520862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a8c25-006a-4a02-bc11-4f66c10a5af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:48.521921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a8c25-006a-4a02-bc11-4f66c10a5af4 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:48.522394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a8c25-006a-4a02-bc11-4f66c10a5af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:48.523224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a8c25-006a-4a02-bc11-4f66c10a5af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:48.528063 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-8b0a8c25-006a-4a02-bc11-4f66c10a5af4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:48.639194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a8c25-006a-4a02-bc11-4f66c10a5af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:48.642147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a8c25-006a-4a02-bc11-4f66c10a5af4 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:48.664295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1020/2036] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:58.158851 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fa05a-2dc8-4346-bc84-f94ac5e67bac service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:58.221256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fa05a-2dc8-4346-bc84-f94ac5e67bac service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:58.226001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:58.226902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:58.227181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:58.227454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] PATH_INFO: `/v3/users/2eee74ae57f44c84bbaa7f085dce5520` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:58.228647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eee74ae57f44c84bbaa7f085dce5520)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:58.231143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:58.281834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eee74ae57f44c84bbaa7f085dce5520'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:58.283604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eee74ae57f44c84bbaa7f085dce5520'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:58.294404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eee74ae57f44c84bbaa7f085dce5520'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:58.295115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eee74ae57f44c84bbaa7f085dce5520'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:58.301846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eee74ae57f44c84bbaa7f085dce5520'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:58.319514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1017/2037] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:58 2022] DELETE /identity/v3/users/2eee74ae57f44c84bbaa7f085dce5520 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:58.324161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9f9e9-7be7-494e-8717-49a61d9c9424 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:58.324692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9f9e9-7be7-494e-8717-49a61d9c9424 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:58.324692 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9f9e9-7be7-494e-8717-49a61d9c9424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:58.342237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-c9c9f9e9-7be7-494e-8717-49a61d9c9424 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:58.343517 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c9c9f9e9-7be7-494e-8717-49a61d9c9424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:58.405195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1021/2038] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:58.415263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:58.416741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:58.491483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b1cea-a182-4423-80ca-3b9b32672190 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:58.495375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292d0bf-a393-4358-8785-1a663b1bac15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:58.496046 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292d0bf-a393-4358-8785-1a663b1bac15 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:58.496350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292d0bf-a393-4358-8785-1a663b1bac15 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:58.496606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292d0bf-a393-4358-8785-1a663b1bac15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:58.506265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292d0bf-a393-4358-8785-1a663b1bac15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:58.507791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292d0bf-a393-4358-8785-1a663b1bac15 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:58.518741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1018/2039] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:57:58.804348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9f9e9-7be7-494e-8717-49a61d9c9424 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:58.806258 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c9c9f9e9-7be7-494e-8717-49a61d9c9424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:58.818893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9f9e9-7be7-494e-8717-49a61d9c9424 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:58.824174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:58.825141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:58.825420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:58.825695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] PATH_INFO: `/v3/projects/6840ff440be646858d4cfb6ec3fe9760` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:58.826956 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6840ff440be646858d4cfb6ec3fe9760)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:58.828870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:58.879393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6840ff440be646858d4cfb6ec3fe9760', 'request_id': 'req-f9d0ce50-7ae1-4df1-abd8-789dffef177b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:58.888938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6840ff440be646858d4cfb6ec3fe9760', 'request_id': 'req-f9d0ce50-7ae1-4df1-abd8-789dffef177b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:58.898238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6840ff440be646858d4cfb6ec3fe9760', 'request_id': 'req-f9d0ce50-7ae1-4df1-abd8-789dffef177b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:58.911436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] The token cache is being invalidate because project 6840ff440be646858d4cfb6ec3fe9760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:58.911710 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:58.912400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:58.913961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1022/2040] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:58 2022] DELETE /identity/v3/projects/6840ff440be646858d4cfb6ec3fe9760 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:58.918204 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292d0bf-a393-4358-8785-1a663b1bac15 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:58.928025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292d0bf-a393-4358-8785-1a663b1bac15 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:58.984205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:58.985088 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:58.985365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:58.985635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] PATH_INFO: `/v3/users/fe8cb9e8af0e4b5c9612a63981bfee47` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:58.986881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe8cb9e8af0e4b5c9612a63981bfee47)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:58.988810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:59.051977 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe8cb9e8af0e4b5c9612a63981bfee47'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:59.052731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe8cb9e8af0e4b5c9612a63981bfee47'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:59.065686 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe8cb9e8af0e4b5c9612a63981bfee47'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:59.066461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe8cb9e8af0e4b5c9612a63981bfee47'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:59.074205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4e9cb924-d531-4975-9135-9093cd84bddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe8cb9e8af0e4b5c9612a63981bfee47'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:59.087254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1019/2041] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:57:58 2022] DELETE /identity/v3/users/fe8cb9e8af0e4b5c9612a63981bfee47 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:59.350486 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:59.352019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:59.373022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ce50-7ae1-4df1-abd8-789dffef177b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:59.431766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:59.433370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:59.433857 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:59.434268 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] PATH_INFO: `/v3/projects/19df30d3301a4338b5c551e7fc40c251` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:59.436009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19df30d3301a4338b5c551e7fc40c251)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:59.437913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:59.483628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19df30d3301a4338b5c551e7fc40c251', 'request_id': 'req-b3cc773a-901c-4902-b8ef-533c76b99783'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:59.489018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19df30d3301a4338b5c551e7fc40c251', 'request_id': 'req-b3cc773a-901c-4902-b8ef-533c76b99783'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:59.496894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19df30d3301a4338b5c551e7fc40c251', 'request_id': 'req-b3cc773a-901c-4902-b8ef-533c76b99783'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:59.510234 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] The token cache is being invalidate because project 19df30d3301a4338b5c551e7fc40c251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:57:59.510520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:59.511262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b3cc773a-901c-4902-b8ef-533c76b99783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:57:59.512815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1023/2042] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:57:59 2022] DELETE /identity/v3/projects/19df30d3301a4338b5c551e7fc40c251 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:59.517600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132fc12-d7a6-4639-b180-3eda9a377b3f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:59.518091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132fc12-d7a6-4639-b180-3eda9a377b3f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:59.518184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132fc12-d7a6-4639-b180-3eda9a377b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:59.537333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-0132fc12-d7a6-4639-b180-3eda9a377b3f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:59.538534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0132fc12-d7a6-4639-b180-3eda9a377b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:59.598577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1020/2043] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:59.607304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ae9b4-a776-42d5-b639-238991eab708 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:59.607684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ae9b4-a776-42d5-b639-238991eab708 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:59.608037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ae9b4-a776-42d5-b639-238991eab708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:59.635196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-901ae9b4-a776-42d5-b639-238991eab708 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:57:59.636471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-901ae9b4-a776-42d5-b639-238991eab708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:57:59.644890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1024/2044] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:57:59.649149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132fc12-d7a6-4639-b180-3eda9a377b3f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:59.659189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132fc12-d7a6-4639-b180-3eda9a377b3f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:59.664184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8385e0-e654-4e77-b154-1d9504a578b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:59.664849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8385e0-e654-4e77-b154-1d9504a578b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:59.665248 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8385e0-e654-4e77-b154-1d9504a578b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:59.665401 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8385e0-e654-4e77-b154-1d9504a578b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:59.666128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8385e0-e654-4e77-b154-1d9504a578b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:57:59.666432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8385e0-e654-4e77-b154-1d9504a578b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:59.668591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8385e0-e654-4e77-b154-1d9504a578b8 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:59.681638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1021/2045] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:57:59.686515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ae9b4-a776-42d5-b639-238991eab708 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:59.698284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ae9b4-a776-42d5-b639-238991eab708 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:59.708874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ce141-11cd-48a0-8907-22cf27669b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:59.709932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ce141-11cd-48a0-8907-22cf27669b07 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:59.710446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ce141-11cd-48a0-8907-22cf27669b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:59.710817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ce141-11cd-48a0-8907-22cf27669b07 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:59.711967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ce141-11cd-48a0-8907-22cf27669b07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:59.715174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ce141-11cd-48a0-8907-22cf27669b07 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:59.743633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1025/2046] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:57:59 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:59.748424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8385e0-e654-4e77-b154-1d9504a578b8 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:59.758632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8385e0-e654-4e77-b154-1d9504a578b8 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:59.821827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454fb7e-6b63-4db1-9844-7444f04b947b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:59.822520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454fb7e-6b63-4db1-9844-7444f04b947b admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:59.822835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454fb7e-6b63-4db1-9844-7444f04b947b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:59.823116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454fb7e-6b63-4db1-9844-7444f04b947b admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:59.823866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454fb7e-6b63-4db1-9844-7444f04b947b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:59.825808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454fb7e-6b63-4db1-9844-7444f04b947b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:59.849049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1022/2047] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:57:59 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:57:59.853127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ce141-11cd-48a0-8907-22cf27669b07 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:59.864125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ce141-11cd-48a0-8907-22cf27669b07 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:59.869264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f318e-bcc1-4515-8dd4-6e5559e4fd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:59.869930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f318e-bcc1-4515-8dd4-6e5559e4fd73 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:59.870288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f318e-bcc1-4515-8dd4-6e5559e4fd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:59.870529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f318e-bcc1-4515-8dd4-6e5559e4fd73 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:59.871331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f318e-bcc1-4515-8dd4-6e5559e4fd73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:59.872923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f318e-bcc1-4515-8dd4-6e5559e4fd73 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:59.883624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1026/2048] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:57:59.887933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454fb7e-6b63-4db1-9844-7444f04b947b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:59.900589 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454fb7e-6b63-4db1-9844-7444f04b947b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:59.906179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f494cb-8d1d-4b52-8c64-f2bf81769f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:59.906930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f494cb-8d1d-4b52-8c64-f2bf81769f16 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:59.907210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f494cb-8d1d-4b52-8c64-f2bf81769f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:59.907483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f494cb-8d1d-4b52-8c64-f2bf81769f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:59.908218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f494cb-8d1d-4b52-8c64-f2bf81769f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:59.909835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f494cb-8d1d-4b52-8c64-f2bf81769f16 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:59.921835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1023/2049] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:57:59.926069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f318e-bcc1-4515-8dd4-6e5559e4fd73 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:57:59.936827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f318e-bcc1-4515-8dd4-6e5559e4fd73 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:57:59.941541 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcd6c3-b167-46e9-a968-d075cf0cbbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:57:59.942423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcd6c3-b167-46e9-a968-d075cf0cbbac admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:59.942731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcd6c3-b167-46e9-a968-d075cf0cbbac admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:59.943011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcd6c3-b167-46e9-a968-d075cf0cbbac admin admin] PATH_INFO: `/v3/projects/3d061ed699cd431c9549293598485af0/users/d83ed344dfd7401283d4066828af52b3/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:57:59.958066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcd6c3-b167-46e9-a968-d075cf0cbbac admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d061ed699cd431c9549293598485af0, user_id=d83ed344dfd7401283d4066828af52b3, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:57:59.960880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcd6c3-b167-46e9-a968-d075cf0cbbac admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:57:59.975731 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1027/2050] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:57:59 2022] PUT /identity/v3/projects/3d061ed699cd431c9549293598485af0/users/d83ed344dfd7401283d4066828af52b3/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:57:59.988507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb8b3-94a3-40b6-9dba-2746da4d1c48 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:57:59.988805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb8b3-94a3-40b6-9dba-2746da4d1c48 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:57:59.989087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb8b3-94a3-40b6-9dba-2746da4d1c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:00.016678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-5a7eb8b3-94a3-40b6-9dba-2746da4d1c48 None None] MFA Rules not processed for user `d83ed344dfd7401283d4066828af52b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:00.017905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5a7eb8b3-94a3-40b6-9dba-2746da4d1c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:00.085704 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1024/2051] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:57:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:00.097551 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcd6c3-b167-46e9-a968-d075cf0cbbac admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:00.099317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-77bcd6c3-b167-46e9-a968-d075cf0cbbac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:00.186987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcd6c3-b167-46e9-a968-d075cf0cbbac admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:00.190840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a48c-0c42-4f16-acdf-16f71196de59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:00.191490 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a48c-0c42-4f16-acdf-16f71196de59 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:00.191906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a48c-0c42-4f16-acdf-16f71196de59 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:00.192195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a48c-0c42-4f16-acdf-16f71196de59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:00.201067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7a48c-0c42-4f16-acdf-16f71196de59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:00.202521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7a48c-0c42-4f16-acdf-16f71196de59 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:00.211476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1028/2052] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:06.075986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef57e5-9feb-4599-8120-11676e64cdc1 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:06.076600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef57e5-9feb-4599-8120-11676e64cdc1 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:06.076712 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef57e5-9feb-4599-8120-11676e64cdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:06.112886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c1ef57e5-9feb-4599-8120-11676e64cdc1 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:06.114143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c1ef57e5-9feb-4599-8120-11676e64cdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:06.180938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1025/2053] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 13:58:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:06.224634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a48c-0c42-4f16-acdf-16f71196de59 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:06.230080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1af7a48c-0c42-4f16-acdf-16f71196de59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:06.317491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a48c-0c42-4f16-acdf-16f71196de59 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:06.324206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c38c2-4044-4fdb-a69e-13374e3d31d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:06.325022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c38c2-4044-4fdb-a69e-13374e3d31d4 service swift] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:06.325459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c38c2-4044-4fdb-a69e-13374e3d31d4 service swift] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:06.325869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c38c2-4044-4fdb-a69e-13374e3d31d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:06.335574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c38c2-4044-4fdb-a69e-13374e3d31d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:06.337189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c38c2-4044-4fdb-a69e-13374e3d31d4 service swift] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:06.347178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1029/2054] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 13:58:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:58:30.291711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef57e5-9feb-4599-8120-11676e64cdc1 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:30.293481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c1ef57e5-9feb-4599-8120-11676e64cdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:30.418841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef57e5-9feb-4599-8120-11676e64cdc1 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:30.434458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:30.435835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:30.436378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:30.437756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] PATH_INFO: `/v3/users/04fdd1810ec24077a7dc4d34759bc801` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:30.439380 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=04fdd1810ec24077a7dc4d34759bc801)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:30.442053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:30.506460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04fdd1810ec24077a7dc4d34759bc801'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:30.507886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04fdd1810ec24077a7dc4d34759bc801'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:30.522739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04fdd1810ec24077a7dc4d34759bc801'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:30.528417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04fdd1810ec24077a7dc4d34759bc801'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:30.537857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04fdd1810ec24077a7dc4d34759bc801'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:30.554830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1026/2055] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:30 2022] DELETE /identity/v3/users/04fdd1810ec24077a7dc4d34759bc801 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:30.559894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325eb99-e610-4096-bd45-fb8cfaccc1df None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:30.560530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325eb99-e610-4096-bd45-fb8cfaccc1df None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:30.560530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325eb99-e610-4096-bd45-fb8cfaccc1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:30.580267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e325eb99-e610-4096-bd45-fb8cfaccc1df None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:30.581466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e325eb99-e610-4096-bd45-fb8cfaccc1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:30.650977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1030/2056] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:30.661038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:30.662498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:30.748674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf91467-6147-49e5-bfd5-cbd66ed464fc admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:30.752698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366582a0-aeea-464a-9ad0-8340ea9502ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:30.753353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366582a0-aeea-464a-9ad0-8340ea9502ca service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:30.753632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366582a0-aeea-464a-9ad0-8340ea9502ca service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:30.753906 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366582a0-aeea-464a-9ad0-8340ea9502ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:30.763725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366582a0-aeea-464a-9ad0-8340ea9502ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:30.765151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366582a0-aeea-464a-9ad0-8340ea9502ca service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:30.775360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1027/2057] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:31.151036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325eb99-e610-4096-bd45-fb8cfaccc1df None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:31.152869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e325eb99-e610-4096-bd45-fb8cfaccc1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:31.167400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325eb99-e610-4096-bd45-fb8cfaccc1df None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:31.173121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:31.173994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:31.174271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:31.174545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] PATH_INFO: `/v3/projects/976b5c9e03624e1797866400092aa757` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:31.175773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=976b5c9e03624e1797866400092aa757)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:31.177719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:31.229102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '976b5c9e03624e1797866400092aa757', 'request_id': 'req-5810723c-21b9-4b2a-abed-adc2c0d25dfb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:31.234334 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '976b5c9e03624e1797866400092aa757', 'request_id': 'req-5810723c-21b9-4b2a-abed-adc2c0d25dfb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:31.242549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '976b5c9e03624e1797866400092aa757', 'request_id': 'req-5810723c-21b9-4b2a-abed-adc2c0d25dfb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:31.255091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] The token cache is being invalidate because project 976b5c9e03624e1797866400092aa757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:31.255373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:31.256016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:31.257554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1031/2058] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:31 2022] DELETE /identity/v3/projects/976b5c9e03624e1797866400092aa757 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:31.262291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366582a0-aeea-464a-9ad0-8340ea9502ca service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:31.272472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366582a0-aeea-464a-9ad0-8340ea9502ca service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:31.333678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:31.334552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:31.334872 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:31.335149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] PATH_INFO: `/v3/users/f3068a154e0d4c1dbeed3b8b8840793b` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:31.336359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3068a154e0d4c1dbeed3b8b8840793b)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:31.339036 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:31.402437 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3068a154e0d4c1dbeed3b8b8840793b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:31.403502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3068a154e0d4c1dbeed3b8b8840793b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:31.420727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3068a154e0d4c1dbeed3b8b8840793b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:31.421676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3068a154e0d4c1dbeed3b8b8840793b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:31.434344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-35f40373-30c2-410d-a851-802285c43b55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3068a154e0d4c1dbeed3b8b8840793b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:31.454366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1028/2059] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:31 2022] DELETE /identity/v3/users/f3068a154e0d4c1dbeed3b8b8840793b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:31.936635 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:31.938169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:31.954780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810723c-21b9-4b2a-abed-adc2c0d25dfb admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:32.055425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:32.057221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.057504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.057778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] PATH_INFO: `/v3/projects/eb9722dd4eb044ba89c53081ba4104eb` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.059145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9722dd4eb044ba89c53081ba4104eb)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:32.061089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:32.136667 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9722dd4eb044ba89c53081ba4104eb', 'request_id': 'req-b1a073f4-8964-40ee-b200-deb756ae80b3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:32.144497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9722dd4eb044ba89c53081ba4104eb', 'request_id': 'req-b1a073f4-8964-40ee-b200-deb756ae80b3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:32.161151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9722dd4eb044ba89c53081ba4104eb', 'request_id': 'req-b1a073f4-8964-40ee-b200-deb756ae80b3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:32.176311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] The token cache is being invalidate because project eb9722dd4eb044ba89c53081ba4104eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:32.176702 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:32.177433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b1a073f4-8964-40ee-b200-deb756ae80b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:32.179865 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1032/2060] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:31 2022] DELETE /identity/v3/projects/eb9722dd4eb044ba89c53081ba4104eb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:32.189174 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b5434-0d2e-4fdf-b94e-5c706f337d8f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.189795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b5434-0d2e-4fdf-b94e-5c706f337d8f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.190783 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b5434-0d2e-4fdf-b94e-5c706f337d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.224149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-574b5434-0d2e-4fdf-b94e-5c706f337d8f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:32.226195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-574b5434-0d2e-4fdf-b94e-5c706f337d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:32.298053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1029/2061] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:32.309233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829b7ef-f57e-4d05-8acb-b146f10fc6ee None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.309599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829b7ef-f57e-4d05-8acb-b146f10fc6ee None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.310173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829b7ef-f57e-4d05-8acb-b146f10fc6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.335262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a829b7ef-f57e-4d05-8acb-b146f10fc6ee None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:32.336980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a829b7ef-f57e-4d05-8acb-b146f10fc6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:32.349023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1033/2062] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:32.353351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b5434-0d2e-4fdf-b94e-5c706f337d8f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:32.363562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b5434-0d2e-4fdf-b94e-5c706f337d8f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:32.369685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72308bf7-6d77-4b01-baf4-6be425668f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:32.370348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72308bf7-6d77-4b01-baf4-6be425668f46 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.370840 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72308bf7-6d77-4b01-baf4-6be425668f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.371290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72308bf7-6d77-4b01-baf4-6be425668f46 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.372035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72308bf7-6d77-4b01-baf4-6be425668f46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:58:32.372331 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72308bf7-6d77-4b01-baf4-6be425668f46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:32.373949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72308bf7-6d77-4b01-baf4-6be425668f46 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:32.391042 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1030/2063] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:58:32.392281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829b7ef-f57e-4d05-8acb-b146f10fc6ee None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:32.405916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829b7ef-f57e-4d05-8acb-b146f10fc6ee None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:32.413233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9448ae6-e652-4756-97d3-97b40da40a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:32.414278 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9448ae6-e652-4756-97d3-97b40da40a8b admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.414862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9448ae6-e652-4756-97d3-97b40da40a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.416257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9448ae6-e652-4756-97d3-97b40da40a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.416958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9448ae6-e652-4756-97d3-97b40da40a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:32.419330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9448ae6-e652-4756-97d3-97b40da40a8b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:32.446733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1034/2064] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:58:32 2022] POST /identity/v3/projects => generated 401 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:32.450897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72308bf7-6d77-4b01-baf4-6be425668f46 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:32.463324 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72308bf7-6d77-4b01-baf4-6be425668f46 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:32.527170 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f48ea-a989-4146-b5be-ce0e878be00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:32.527877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f48ea-a989-4146-b5be-ce0e878be00f admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.528153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f48ea-a989-4146-b5be-ce0e878be00f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.528437 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f48ea-a989-4146-b5be-ce0e878be00f admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.529142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f48ea-a989-4146-b5be-ce0e878be00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:32.531113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f48ea-a989-4146-b5be-ce0e878be00f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:32.552740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1031/2065] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:58:32 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:32.559166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9448ae6-e652-4756-97d3-97b40da40a8b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:32.569656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9448ae6-e652-4756-97d3-97b40da40a8b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:32.576642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41913a16-e45b-46ab-8121-24ad4fd3f457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:32.577318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41913a16-e45b-46ab-8121-24ad4fd3f457 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.577597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41913a16-e45b-46ab-8121-24ad4fd3f457 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.577880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41913a16-e45b-46ab-8121-24ad4fd3f457 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.578676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41913a16-e45b-46ab-8121-24ad4fd3f457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:32.580325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41913a16-e45b-46ab-8121-24ad4fd3f457 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:32.594645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1035/2066] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:32.595982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f48ea-a989-4146-b5be-ce0e878be00f admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:32.607970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f48ea-a989-4146-b5be-ce0e878be00f admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:32.616040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b786d90-41f8-4a4e-b30e-c4fc45b7a2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:32.616040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b786d90-41f8-4a4e-b30e-c4fc45b7a2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.616040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b786d90-41f8-4a4e-b30e-c4fc45b7a2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.616040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b786d90-41f8-4a4e-b30e-c4fc45b7a2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.616647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b786d90-41f8-4a4e-b30e-c4fc45b7a2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:32.617872 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b786d90-41f8-4a4e-b30e-c4fc45b7a2d9 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:32.631062 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1032/2067] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:32.635746 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41913a16-e45b-46ab-8121-24ad4fd3f457 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:32.647950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41913a16-e45b-46ab-8121-24ad4fd3f457 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:32.653532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e0de9-4c0b-46fe-a9e4-9e21e4db511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:32.654825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0de9-4c0b-46fe-a9e4-9e21e4db511a admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.655283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0de9-4c0b-46fe-a9e4-9e21e4db511a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.655563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0de9-4c0b-46fe-a9e4-9e21e4db511a admin admin] PATH_INFO: `/v3/projects/9d373d27e9484dc1bf5c128d006c7546/users/c5c57f0f981a4d419abfbada0d21fb7f/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.682691 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e0de9-4c0b-46fe-a9e4-9e21e4db511a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d373d27e9484dc1bf5c128d006c7546, user_id=c5c57f0f981a4d419abfbada0d21fb7f, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:32.686525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e0de9-4c0b-46fe-a9e4-9e21e4db511a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:32.705002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1036/2068] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:32 2022] PUT /identity/v3/projects/9d373d27e9484dc1bf5c128d006c7546/users/c5c57f0f981a4d419abfbada0d21fb7f/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:32.716842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7462e84-5dc6-4109-a0e6-aaf023ce4d71 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.717391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7462e84-5dc6-4109-a0e6-aaf023ce4d71 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.717896 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7462e84-5dc6-4109-a0e6-aaf023ce4d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.751707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a7462e84-5dc6-4109-a0e6-aaf023ce4d71 None None] MFA Rules not processed for user `c5c57f0f981a4d419abfbada0d21fb7f`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:32.753020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a7462e84-5dc6-4109-a0e6-aaf023ce4d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:32.844490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1033/2069] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:32.848970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e0de9-4c0b-46fe-a9e4-9e21e4db511a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:32.972222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e0de9-4c0b-46fe-a9e4-9e21e4db511a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:32.986994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e9b1a-00f2-47d4-bb2f-b18bbd865785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:32.987953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e9b1a-00f2-47d4-bb2f-b18bbd865785 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:32.988859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e9b1a-00f2-47d4-bb2f-b18bbd865785 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:32.989134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e9b1a-00f2-47d4-bb2f-b18bbd865785 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:32.989860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e9b1a-00f2-47d4-bb2f-b18bbd865785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:32.993220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e9b1a-00f2-47d4-bb2f-b18bbd865785 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:33.035441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1037/2070] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:58:32 2022] POST /identity/v3/projects => generated 401 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:58:33.036353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7462e84-5dc6-4109-a0e6-aaf023ce4d71 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:33.052413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7462e84-5dc6-4109-a0e6-aaf023ce4d71 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:33.184930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b6e6a-79d9-486a-b7ef-723ce1f10a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:33.185949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b6e6a-79d9-486a-b7ef-723ce1f10a5e admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:33.186398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b6e6a-79d9-486a-b7ef-723ce1f10a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:33.186865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b6e6a-79d9-486a-b7ef-723ce1f10a5e admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:33.187574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b6e6a-79d9-486a-b7ef-723ce1f10a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:33.190519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b6e6a-79d9-486a-b7ef-723ce1f10a5e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:33.211569 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1034/2071] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:58:33 2022] POST /identity/v3/users => generated 307 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:58:33.218786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e9b1a-00f2-47d4-bb2f-b18bbd865785 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:33.235364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e9b1a-00f2-47d4-bb2f-b18bbd865785 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:33.248614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c743de-1fe4-4d44-9e61-003b7affdeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:33.249903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c743de-1fe4-4d44-9e61-003b7affdeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:33.250797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c743de-1fe4-4d44-9e61-003b7affdeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:33.251232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c743de-1fe4-4d44-9e61-003b7affdeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:33.260900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c743de-1fe4-4d44-9e61-003b7affdeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:33.260900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c743de-1fe4-4d44-9e61-003b7affdeb6 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:33.261998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695badc-c8c2-45c3-9e04-71af6e8f20a3 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:33.262480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695badc-c8c2-45c3-9e04-71af6e8f20a3 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:33.262738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695badc-c8c2-45c3-9e04-71af6e8f20a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:33.266199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1038/2072] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:33.270750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c743de-1fe4-4d44-9e61-003b7affdeb6 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:33.281026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c743de-1fe4-4d44-9e61-003b7affdeb6 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:33.283388 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f695badc-c8c2-45c3-9e04-71af6e8f20a3 None None] MFA Rules not processed for user `648ae92126944b50a22ad780e92df914`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:33.284619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f695badc-c8c2-45c3-9e04-71af6e8f20a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:33.286036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8326fe-c5b3-4fda-ab6b-da4cddcf35c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:33.287036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8326fe-c5b3-4fda-ab6b-da4cddcf35c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:33.287415 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8326fe-c5b3-4fda-ab6b-da4cddcf35c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:33.287699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8326fe-c5b3-4fda-ab6b-da4cddcf35c4 admin admin] PATH_INFO: `/v3/projects/8f87707b1da0406d934800fa54c31e70/users/c2c0188d8ff74d699235bb07b98ac497/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:33.303307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8326fe-c5b3-4fda-ab6b-da4cddcf35c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f87707b1da0406d934800fa54c31e70, user_id=c2c0188d8ff74d699235bb07b98ac497, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:33.307059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8326fe-c5b3-4fda-ab6b-da4cddcf35c4 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:33.330530 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1039/2073] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:33 2022] PUT /identity/v3/projects/8f87707b1da0406d934800fa54c31e70/users/c2c0188d8ff74d699235bb07b98ac497/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:58:33.331600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8326fe-c5b3-4fda-ab6b-da4cddcf35c4 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:33.354166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1035/2074] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:58:33.402176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8326fe-c5b3-4fda-ab6b-da4cddcf35c4 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:33.407318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5cb84-e860-4993-9293-1429f52f9049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:33.407976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5cb84-e860-4993-9293-1429f52f9049 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:33.408317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5cb84-e860-4993-9293-1429f52f9049 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:33.408605 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5cb84-e860-4993-9293-1429f52f9049 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:33.409255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5cb84-e860-4993-9293-1429f52f9049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:33.410881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5cb84-e860-4993-9293-1429f52f9049 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:33.421756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1040/2075] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:33.426111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695badc-c8c2-45c3-9e04-71af6e8f20a3 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:33.438128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695badc-c8c2-45c3-9e04-71af6e8f20a3 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:33.444350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4270358-0515-4670-896a-d9e3848e235d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:33.445424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4270358-0515-4670-896a-d9e3848e235d admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:33.445826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4270358-0515-4670-896a-d9e3848e235d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:33.446220 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4270358-0515-4670-896a-d9e3848e235d admin admin] PATH_INFO: `/v3/projects/8f87707b1da0406d934800fa54c31e70/users/c2c0188d8ff74d699235bb07b98ac497/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:33.448960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4270358-0515-4670-896a-d9e3848e235d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f87707b1da0406d934800fa54c31e70, user_id=c2c0188d8ff74d699235bb07b98ac497, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:33.451707 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4270358-0515-4670-896a-d9e3848e235d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:33.463670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1036/2076] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:33 2022] PUT /identity/v3/projects/8f87707b1da0406d934800fa54c31e70/users/c2c0188d8ff74d699235bb07b98ac497/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:33.474507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f524f6-efad-4ca5-929a-1e84f8388bbd None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:33.474789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f524f6-efad-4ca5-929a-1e84f8388bbd None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:33.475038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f524f6-efad-4ca5-929a-1e84f8388bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:33.506399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-c8f524f6-efad-4ca5-929a-1e84f8388bbd None None] MFA Rules not processed for user `c2c0188d8ff74d699235bb07b98ac497`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:33.507804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c8f524f6-efad-4ca5-929a-1e84f8388bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:33.591212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1041/2077] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:33 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:58:33.604062 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4270358-0515-4670-896a-d9e3848e235d admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:33.612781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d4270358-0515-4670-896a-d9e3848e235d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:33.686255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4270358-0515-4670-896a-d9e3848e235d admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:33.692560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e48d1b-8d5a-47ba-bb15-c2d2eb29d1f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:33.693215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48d1b-8d5a-47ba-bb15-c2d2eb29d1f4 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:33.693841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48d1b-8d5a-47ba-bb15-c2d2eb29d1f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:33.694252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48d1b-8d5a-47ba-bb15-c2d2eb29d1f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:33.705192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e48d1b-8d5a-47ba-bb15-c2d2eb29d1f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:33.706783 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e48d1b-8d5a-47ba-bb15-c2d2eb29d1f4 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:33.721692 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1037/2078] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:33.812906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f524f6-efad-4ca5-929a-1e84f8388bbd None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:33.827466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f524f6-efad-4ca5-929a-1e84f8388bbd None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:33.831703 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d5cdb-6e97-446b-a119-9caf8e228598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:33.832353 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d5cdb-6e97-446b-a119-9caf8e228598 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:33.832631 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d5cdb-6e97-446b-a119-9caf8e228598 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:33.832917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d5cdb-6e97-446b-a119-9caf8e228598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:33.897987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d5cdb-6e97-446b-a119-9caf8e228598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:33.899538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d5cdb-6e97-446b-a119-9caf8e228598 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:33.922878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1042/2079] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:34.371045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e48d1b-8d5a-47ba-bb15-c2d2eb29d1f4 service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:34.372523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c7e48d1b-8d5a-47ba-bb15-c2d2eb29d1f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:34.437240 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e48d1b-8d5a-47ba-bb15-c2d2eb29d1f4 service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:34.441843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:34.442681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:34.443002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:34.443276 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] PATH_INFO: `/v3/users/d83ed344dfd7401283d4066828af52b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:34.444468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d83ed344dfd7401283d4066828af52b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:34.447020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:34.491392 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd83ed344dfd7401283d4066828af52b3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:34.492104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd83ed344dfd7401283d4066828af52b3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:34.504269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd83ed344dfd7401283d4066828af52b3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:34.504976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd83ed344dfd7401283d4066828af52b3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:34.512387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd83ed344dfd7401283d4066828af52b3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:34.525863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1038/2080] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:34 2022] DELETE /identity/v3/users/d83ed344dfd7401283d4066828af52b3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:34.530966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d4d7-74dc-4a9f-b703-1f6d653971ce None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:34.531499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d4d7-74dc-4a9f-b703-1f6d653971ce None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:34.531499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d4d7-74dc-4a9f-b703-1f6d653971ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:34.550298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-37c0d4d7-74dc-4a9f-b703-1f6d653971ce None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:34.551476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-37c0d4d7-74dc-4a9f-b703-1f6d653971ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:34.626136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1043/2081] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:34.633325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:34.635278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:34.703447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d52b1-dbae-46aa-8c12-fa1a8e151b46 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:34.707134 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46987a78-d2ee-46cd-8b04-a0b8adc5e96f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:34.707886 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46987a78-d2ee-46cd-8b04-a0b8adc5e96f service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:34.708346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46987a78-d2ee-46cd-8b04-a0b8adc5e96f service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:34.708514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46987a78-d2ee-46cd-8b04-a0b8adc5e96f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:34.722386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46987a78-d2ee-46cd-8b04-a0b8adc5e96f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:34.723903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46987a78-d2ee-46cd-8b04-a0b8adc5e96f service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:34.734385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1039/2082] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:58:35.055341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0d4d7-74dc-4a9f-b703-1f6d653971ce None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:35.057023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-37c0d4d7-74dc-4a9f-b703-1f6d653971ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:35.079292 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0d4d7-74dc-4a9f-b703-1f6d653971ce None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:35.086622 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:35.087732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:35.088094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:35.088700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] PATH_INFO: `/v3/projects/3d061ed699cd431c9549293598485af0` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:35.090479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d061ed699cd431c9549293598485af0)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:35.102460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:35.185080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d061ed699cd431c9549293598485af0', 'request_id': 'req-c2cad066-3909-4b88-b42c-68bc89fe70ee'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:35.192410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d061ed699cd431c9549293598485af0', 'request_id': 'req-c2cad066-3909-4b88-b42c-68bc89fe70ee'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:35.202957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d061ed699cd431c9549293598485af0', 'request_id': 'req-c2cad066-3909-4b88-b42c-68bc89fe70ee'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:35.215989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] The token cache is being invalidate because project 3d061ed699cd431c9549293598485af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:35.216394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:35.217183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c2cad066-3909-4b88-b42c-68bc89fe70ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:35.218846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1044/2083] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:35 2022] DELETE /identity/v3/projects/3d061ed699cd431c9549293598485af0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:38.469874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46987a78-d2ee-46cd-8b04-a0b8adc5e96f service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:38.472117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-46987a78-d2ee-46cd-8b04-a0b8adc5e96f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:38.486759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46987a78-d2ee-46cd-8b04-a0b8adc5e96f service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:38.554944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:38.556648 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:38.556944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:38.557386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] PATH_INFO: `/v3/users/c5c57f0f981a4d419abfbada0d21fb7f` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:38.558811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c57f0f981a4d419abfbada0d21fb7f)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:38.560923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:38.619058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c57f0f981a4d419abfbada0d21fb7f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:38.619744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c57f0f981a4d419abfbada0d21fb7f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:38.632489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c57f0f981a4d419abfbada0d21fb7f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:38.633249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c57f0f981a4d419abfbada0d21fb7f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:38.642610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c57f0f981a4d419abfbada0d21fb7f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:38.656544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1040/2084] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:38 2022] DELETE /identity/v3/users/c5c57f0f981a4d419abfbada0d21fb7f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:38.661311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b6f0-a588-4e9c-9f96-546266b3e355 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:38.661844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b6f0-a588-4e9c-9f96-546266b3e355 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:38.661844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b6f0-a588-4e9c-9f96-546266b3e355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:38.680560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2810b6f0-a588-4e9c-9f96-546266b3e355 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:38.682523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2810b6f0-a588-4e9c-9f96-546266b3e355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:38.757697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1045/2085] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:38.769372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:38.770899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:38.859360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e0b5d-bf32-4c01-86c4-232f4fe78c1a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:38.867106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f727f3-bef6-46b4-8e34-0405a360e4c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:38.868336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f727f3-bef6-46b4-8e34-0405a360e4c5 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:38.868805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f727f3-bef6-46b4-8e34-0405a360e4c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:38.869446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f727f3-bef6-46b4-8e34-0405a360e4c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:38.882292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f727f3-bef6-46b4-8e34-0405a360e4c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:38.884272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f727f3-bef6-46b4-8e34-0405a360e4c5 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:38.896315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1041/2086] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:39.471962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810b6f0-a588-4e9c-9f96-546266b3e355 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:39.473487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2810b6f0-a588-4e9c-9f96-546266b3e355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:39.495477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810b6f0-a588-4e9c-9f96-546266b3e355 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:39.501537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:39.502426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:39.502723 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:39.503005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] PATH_INFO: `/v3/projects/9d373d27e9484dc1bf5c128d006c7546` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:39.504285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d373d27e9484dc1bf5c128d006c7546)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:39.507835 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:39.588342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d373d27e9484dc1bf5c128d006c7546', 'request_id': 'req-22db41b3-7529-47f4-85ad-4a948979d61c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:39.601716 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d373d27e9484dc1bf5c128d006c7546', 'request_id': 'req-22db41b3-7529-47f4-85ad-4a948979d61c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:39.612634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d373d27e9484dc1bf5c128d006c7546', 'request_id': 'req-22db41b3-7529-47f4-85ad-4a948979d61c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:39.629431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] The token cache is being invalidate because project 9d373d27e9484dc1bf5c128d006c7546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:39.629734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:39.630485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:39.632480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1046/2087] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:39 2022] DELETE /identity/v3/projects/9d373d27e9484dc1bf5c128d006c7546 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:39.636988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f727f3-bef6-46b4-8e34-0405a360e4c5 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:39.651177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f727f3-bef6-46b4-8e34-0405a360e4c5 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:39.764670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:39.765814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:39.766415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:39.766885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] PATH_INFO: `/v3/users/c2c0188d8ff74d699235bb07b98ac497` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:39.768317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c0188d8ff74d699235bb07b98ac497)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:39.770427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:39.841081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c0188d8ff74d699235bb07b98ac497'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:39.842014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c0188d8ff74d699235bb07b98ac497'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:39.856519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c0188d8ff74d699235bb07b98ac497'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:39.857352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c0188d8ff74d699235bb07b98ac497'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:39.865083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-2c392ac3-38fd-495f-813c-f5a9f103f06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c0188d8ff74d699235bb07b98ac497'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:39.879530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1042/2088] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:39 2022] DELETE /identity/v3/users/c2c0188d8ff74d699235bb07b98ac497 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:58:40.263974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:40.265729 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:40.279010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db41b3-7529-47f4-85ad-4a948979d61c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:40.339156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:40.340179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:40.340465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:40.340743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] PATH_INFO: `/v3/projects/8f87707b1da0406d934800fa54c31e70` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:40.341937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f87707b1da0406d934800fa54c31e70)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:40.343889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:40.416502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f87707b1da0406d934800fa54c31e70', 'request_id': 'req-80942f39-6609-4901-9e88-b80ed1f95b6b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:40.424388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f87707b1da0406d934800fa54c31e70', 'request_id': 'req-80942f39-6609-4901-9e88-b80ed1f95b6b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:40.436469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f87707b1da0406d934800fa54c31e70', 'request_id': 'req-80942f39-6609-4901-9e88-b80ed1f95b6b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:40.451238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] The token cache is being invalidate because project 8f87707b1da0406d934800fa54c31e70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:40.451752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:40.452486 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-80942f39-6609-4901-9e88-b80ed1f95b6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:40.459206 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1047/2089] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:40 2022] DELETE /identity/v3/projects/8f87707b1da0406d934800fa54c31e70 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:40.463792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad7bd9-61c6-4fda-8df0-4813d877b9cf None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:40.464077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad7bd9-61c6-4fda-8df0-4813d877b9cf None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:40.464360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad7bd9-61c6-4fda-8df0-4813d877b9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:40.482279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b6ad7bd9-61c6-4fda-8df0-4813d877b9cf None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:40.483585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b6ad7bd9-61c6-4fda-8df0-4813d877b9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:40.566294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1043/2090] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:58:40.575280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822f180-aa14-49f5-89e7-9c9650be376b None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:40.575818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822f180-aa14-49f5-89e7-9c9650be376b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:40.576188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822f180-aa14-49f5-89e7-9c9650be376b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:40.600562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-5822f180-aa14-49f5-89e7-9c9650be376b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:40.601932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5822f180-aa14-49f5-89e7-9c9650be376b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:40.612989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1048/2091] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:40.617642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad7bd9-61c6-4fda-8df0-4813d877b9cf None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:40.640860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad7bd9-61c6-4fda-8df0-4813d877b9cf None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:40.649216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b874126-bd60-4f91-9e6c-d612ed6a4a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:40.651002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b874126-bd60-4f91-9e6c-d612ed6a4a40 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:40.651353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b874126-bd60-4f91-9e6c-d612ed6a4a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:40.651700 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b874126-bd60-4f91-9e6c-d612ed6a4a40 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:40.652881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b874126-bd60-4f91-9e6c-d612ed6a4a40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:58:40.653259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b874126-bd60-4f91-9e6c-d612ed6a4a40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:40.655007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b874126-bd60-4f91-9e6c-d612ed6a4a40 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:40.674894 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1044/2092] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:58:40.677693 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822f180-aa14-49f5-89e7-9c9650be376b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:40.691184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822f180-aa14-49f5-89e7-9c9650be376b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:40.697041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25f41e-d1fd-4b73-bcbe-91236336e860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:40.697869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25f41e-d1fd-4b73-bcbe-91236336e860 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:40.698366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25f41e-d1fd-4b73-bcbe-91236336e860 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:40.698742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25f41e-d1fd-4b73-bcbe-91236336e860 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:40.699683 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25f41e-d1fd-4b73-bcbe-91236336e860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:40.702533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25f41e-d1fd-4b73-bcbe-91236336e860 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:40.758756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1049/2093] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:58:40 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:58:40.762625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b874126-bd60-4f91-9e6c-d612ed6a4a40 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:40.776145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b874126-bd60-4f91-9e6c-d612ed6a4a40 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:40.901136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7e6d4-3117-486c-9156-b7860d3df4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:40.902046 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7e6d4-3117-486c-9156-b7860d3df4a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:40.902350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7e6d4-3117-486c-9156-b7860d3df4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:40.902777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7e6d4-3117-486c-9156-b7860d3df4a5 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:40.903635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7e6d4-3117-486c-9156-b7860d3df4a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:40.905816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7e6d4-3117-486c-9156-b7860d3df4a5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:40.951701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1045/2094] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:58:40 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:40.952015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25f41e-d1fd-4b73-bcbe-91236336e860 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:40.965833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25f41e-d1fd-4b73-bcbe-91236336e860 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:40.970990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6254f-cbb5-4233-993b-ed19f999c674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:40.976109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6254f-cbb5-4233-993b-ed19f999c674 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:40.976109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6254f-cbb5-4233-993b-ed19f999c674 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:40.976109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6254f-cbb5-4233-993b-ed19f999c674 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:40.976109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6254f-cbb5-4233-993b-ed19f999c674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:40.976109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6254f-cbb5-4233-993b-ed19f999c674 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:41.001438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1050/2095] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:41.001668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7e6d4-3117-486c-9156-b7860d3df4a5 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:41.015906 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7e6d4-3117-486c-9156-b7860d3df4a5 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:41.026380 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64448cb-435b-47ed-8859-4d4972887d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:41.027900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64448cb-435b-47ed-8859-4d4972887d80 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:41.028502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64448cb-435b-47ed-8859-4d4972887d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:41.029272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64448cb-435b-47ed-8859-4d4972887d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:41.030447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64448cb-435b-47ed-8859-4d4972887d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:41.033761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64448cb-435b-47ed-8859-4d4972887d80 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:41.077789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1046/2096] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:41.079789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6254f-cbb5-4233-993b-ed19f999c674 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:41.116843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6254f-cbb5-4233-993b-ed19f999c674 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:41.151368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd08de-0080-4d2e-8b3d-b039eee78c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:41.153342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd08de-0080-4d2e-8b3d-b039eee78c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:41.154222 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd08de-0080-4d2e-8b3d-b039eee78c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:41.155134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd08de-0080-4d2e-8b3d-b039eee78c36 admin admin] PATH_INFO: `/v3/projects/787c362bec724c00af18a7484a2525be/users/240af4307a2a49408b4038556864952d/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:41.177690 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd08de-0080-4d2e-8b3d-b039eee78c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=787c362bec724c00af18a7484a2525be, user_id=240af4307a2a49408b4038556864952d, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:41.180679 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd08de-0080-4d2e-8b3d-b039eee78c36 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:41.239756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1051/2097] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:41 2022] PUT /identity/v3/projects/787c362bec724c00af18a7484a2525be/users/240af4307a2a49408b4038556864952d/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:41.252211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ee97e-3b47-4b33-b5fd-0fcc449e711f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:41.252655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ee97e-3b47-4b33-b5fd-0fcc449e711f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:41.252882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ee97e-3b47-4b33-b5fd-0fcc449e711f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:41.341190 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-369ee97e-3b47-4b33-b5fd-0fcc449e711f None None] MFA Rules not processed for user `240af4307a2a49408b4038556864952d`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:41.343392 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-369ee97e-3b47-4b33-b5fd-0fcc449e711f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:41.499711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1047/2098] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:41.511855 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd08de-0080-4d2e-8b3d-b039eee78c36 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:41.645155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd08de-0080-4d2e-8b3d-b039eee78c36 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:41.664305 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07a011-b4a4-43cf-96b6-7271d557d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:41.665673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07a011-b4a4-43cf-96b6-7271d557d1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:41.666354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07a011-b4a4-43cf-96b6-7271d557d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:41.667678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07a011-b4a4-43cf-96b6-7271d557d1a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:41.669465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07a011-b4a4-43cf-96b6-7271d557d1a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:41.673969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07a011-b4a4-43cf-96b6-7271d557d1a3 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:41.730093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1052/2099] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:58:41 2022] POST /identity/v3/projects => generated 383 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:41.735915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ee97e-3b47-4b33-b5fd-0fcc449e711f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:41.753201 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ee97e-3b47-4b33-b5fd-0fcc449e711f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:41.876293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dd438-a213-463c-913d-170b39f0130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:41.879371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dd438-a213-463c-913d-170b39f0130c admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:41.881390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dd438-a213-463c-913d-170b39f0130c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:41.881883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dd438-a213-463c-913d-170b39f0130c admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:41.885390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dd438-a213-463c-913d-170b39f0130c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:41.889828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dd438-a213-463c-913d-170b39f0130c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:41.966210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07a011-b4a4-43cf-96b6-7271d557d1a3 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:41.980388 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1048/2100] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:58:41 2022] POST /identity/v3/users => generated 298 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:41.993969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07a011-b4a4-43cf-96b6-7271d557d1a3 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:41.999833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac8140-64ff-4042-8c1d-2e9890107d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:42.000768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac8140-64ff-4042-8c1d-2e9890107d9f admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:42.000890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac8140-64ff-4042-8c1d-2e9890107d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:42.001243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac8140-64ff-4042-8c1d-2e9890107d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:42.002338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac8140-64ff-4042-8c1d-2e9890107d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:42.003804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac8140-64ff-4042-8c1d-2e9890107d9f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:42.041355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1053/2101] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:42.046027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dd438-a213-463c-913d-170b39f0130c admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:42.075281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dd438-a213-463c-913d-170b39f0130c admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:42.083343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7bbec-4b04-4715-956f-1e79831d61fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:42.084836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7bbec-4b04-4715-956f-1e79831d61fd admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:42.085225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7bbec-4b04-4715-956f-1e79831d61fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:42.085604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7bbec-4b04-4715-956f-1e79831d61fd admin admin] PATH_INFO: `/v3/projects/ee39471205eb4f1c83e7a34474c5fe06/users/17881e2146e244528dc404950291f6c2/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:42.109689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7bbec-4b04-4715-956f-1e79831d61fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee39471205eb4f1c83e7a34474c5fe06, user_id=17881e2146e244528dc404950291f6c2, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:42.112882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7bbec-4b04-4715-956f-1e79831d61fd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:42.132721 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1049/2102] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:42 2022] PUT /identity/v3/projects/ee39471205eb4f1c83e7a34474c5fe06/users/17881e2146e244528dc404950291f6c2/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:42.137469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac8140-64ff-4042-8c1d-2e9890107d9f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:42.153758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac8140-64ff-4042-8c1d-2e9890107d9f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:42.238603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6a497-76ed-438b-b271-a4755201e917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:42.239738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6a497-76ed-438b-b271-a4755201e917 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:42.240139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6a497-76ed-438b-b271-a4755201e917 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:42.240488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6a497-76ed-438b-b271-a4755201e917 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:42.242256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6a497-76ed-438b-b271-a4755201e917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:42.245531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6a497-76ed-438b-b271-a4755201e917 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:42.261774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1054/2103] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:42.265885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7bbec-4b04-4715-956f-1e79831d61fd admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:42.276981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7bbec-4b04-4715-956f-1e79831d61fd admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:42.284136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0958f74-f07a-4c98-bcb4-f1753aa728ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:42.285132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0958f74-f07a-4c98-bcb4-f1753aa728ff admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:42.285448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0958f74-f07a-4c98-bcb4-f1753aa728ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:42.285788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0958f74-f07a-4c98-bcb4-f1753aa728ff admin admin] PATH_INFO: `/v3/projects/ee39471205eb4f1c83e7a34474c5fe06/users/17881e2146e244528dc404950291f6c2/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:42.287908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0958f74-f07a-4c98-bcb4-f1753aa728ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee39471205eb4f1c83e7a34474c5fe06, user_id=17881e2146e244528dc404950291f6c2, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:42.291541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0958f74-f07a-4c98-bcb4-f1753aa728ff admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:42.311360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1050/2104] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:42 2022] PUT /identity/v3/projects/ee39471205eb4f1c83e7a34474c5fe06/users/17881e2146e244528dc404950291f6c2/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:42.318228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53822313-b4ba-4cf0-a1ab-cc62e92b9bab None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:42.318521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53822313-b4ba-4cf0-a1ab-cc62e92b9bab None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:42.318829 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53822313-b4ba-4cf0-a1ab-cc62e92b9bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:42.348121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-53822313-b4ba-4cf0-a1ab-cc62e92b9bab None None] MFA Rules not processed for user `17881e2146e244528dc404950291f6c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:42.349406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-53822313-b4ba-4cf0-a1ab-cc62e92b9bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:42.425628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1055/2105] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:42.437652 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0958f74-f07a-4c98-bcb4-f1753aa728ff admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:42.439461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e0958f74-f07a-4c98-bcb4-f1753aa728ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:42.508952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0958f74-f07a-4c98-bcb4-f1753aa728ff admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:42.517040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f3da4-b22f-496c-9c8d-ce5405cdc239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:42.518366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f3da4-b22f-496c-9c8d-ce5405cdc239 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:42.518757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f3da4-b22f-496c-9c8d-ce5405cdc239 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:42.519053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f3da4-b22f-496c-9c8d-ce5405cdc239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:42.532693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f3da4-b22f-496c-9c8d-ce5405cdc239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:42.536064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f3da4-b22f-496c-9c8d-ce5405cdc239 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:42.549622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1051/2106] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:42.734048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53822313-b4ba-4cf0-a1ab-cc62e92b9bab None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:42.752431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53822313-b4ba-4cf0-a1ab-cc62e92b9bab None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:42.756598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fe8d7-5342-4b8f-b5e4-f369bed01af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:42.757312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fe8d7-5342-4b8f-b5e4-f369bed01af1 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:42.757600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fe8d7-5342-4b8f-b5e4-f369bed01af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:42.757871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fe8d7-5342-4b8f-b5e4-f369bed01af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:42.832908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fe8d7-5342-4b8f-b5e4-f369bed01af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:42.834818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fe8d7-5342-4b8f-b5e4-f369bed01af1 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:42.851032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1056/2107] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:44.898071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd3f0-bb45-43b0-87d4-2dcc036933e5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:44.898567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd3f0-bb45-43b0-87d4-2dcc036933e5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:44.898663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd3f0-bb45-43b0-87d4-2dcc036933e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:44.949591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-d0bdd3f0-bb45-43b0-87d4-2dcc036933e5 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:44.951709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d0bdd3f0-bb45-43b0-87d4-2dcc036933e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:45.025533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1052/2108] 10.222.0.79 () {58 vars in 967 bytes} [Fri May 13 13:58:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:45.038382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fe8d7-5342-4b8f-b5e4-f369bed01af1 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:45.039899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b96fe8d7-5342-4b8f-b5e4-f369bed01af1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:45.137131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fe8d7-5342-4b8f-b5e4-f369bed01af1 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:45.142818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed826fd-e7b0-4abb-98d4-4f47a5cf9909 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:45.143987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed826fd-e7b0-4abb-98d4-4f47a5cf9909 service swift] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:45.144310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed826fd-e7b0-4abb-98d4-4f47a5cf9909 service swift] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:45.144697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed826fd-e7b0-4abb-98d4-4f47a5cf9909 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:45.153577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed826fd-e7b0-4abb-98d4-4f47a5cf9909 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:45.155215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed826fd-e7b0-4abb-98d4-4f47a5cf9909 service swift] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:45.164355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1057/2109] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 13:58:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:58:45.752072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdd3f0-bb45-43b0-87d4-2dcc036933e5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:45.842109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdd3f0-bb45-43b0-87d4-2dcc036933e5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:45.848728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:45.850390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:45.850899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:45.851450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] PATH_INFO: `/v3/users/240af4307a2a49408b4038556864952d` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:45.853448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=240af4307a2a49408b4038556864952d)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:45.855726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:45.916951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '240af4307a2a49408b4038556864952d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:45.918295 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '240af4307a2a49408b4038556864952d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:45.930759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '240af4307a2a49408b4038556864952d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:45.931523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '240af4307a2a49408b4038556864952d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:45.944211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '240af4307a2a49408b4038556864952d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:45.958344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1053/2110] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:45 2022] DELETE /identity/v3/users/240af4307a2a49408b4038556864952d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:45.963483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1aef9-43c5-48c4-a1e1-530ae1d2b4b4 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:45.963843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1aef9-43c5-48c4-a1e1-530ae1d2b4b4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:45.964197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1aef9-43c5-48c4-a1e1-530ae1d2b4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:45.985045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-cca1aef9-43c5-48c4-a1e1-530ae1d2b4b4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:45.986355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-cca1aef9-43c5-48c4-a1e1-530ae1d2b4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:46.058629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1058/2111] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:46.065077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:46.066498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:46.142964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a160b3-6ca2-464e-a379-8cf09924fd2a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:46.146930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de81f0-a180-4deb-b456-917d094faf07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:46.148016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de81f0-a180-4deb-b456-917d094faf07 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:46.148016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de81f0-a180-4deb-b456-917d094faf07 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:46.149566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de81f0-a180-4deb-b456-917d094faf07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:46.160440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de81f0-a180-4deb-b456-917d094faf07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:46.161888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de81f0-a180-4deb-b456-917d094faf07 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:46.174256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1054/2112] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:46.770284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1aef9-43c5-48c4-a1e1-530ae1d2b4b4 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:46.771784 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-cca1aef9-43c5-48c4-a1e1-530ae1d2b4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:46.789372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1aef9-43c5-48c4-a1e1-530ae1d2b4b4 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:46.798178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:46.799126 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:46.799414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:46.799689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] PATH_INFO: `/v3/projects/787c362bec724c00af18a7484a2525be` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:46.801671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=787c362bec724c00af18a7484a2525be)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:46.804495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:46.857684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '787c362bec724c00af18a7484a2525be', 'request_id': 'req-2632aaa5-9f85-4517-957d-f5f6b8ceb902'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:46.863069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '787c362bec724c00af18a7484a2525be', 'request_id': 'req-2632aaa5-9f85-4517-957d-f5f6b8ceb902'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:46.872221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '787c362bec724c00af18a7484a2525be', 'request_id': 'req-2632aaa5-9f85-4517-957d-f5f6b8ceb902'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:46.884251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] The token cache is being invalidate because project 787c362bec724c00af18a7484a2525be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:46.884559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:46.885665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:46.887720 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1059/2113] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:46 2022] DELETE /identity/v3/projects/787c362bec724c00af18a7484a2525be => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:46.891821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de81f0-a180-4deb-b456-917d094faf07 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:46.902447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de81f0-a180-4deb-b456-917d094faf07 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:46.965115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:46.965995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:46.966271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:46.966545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] PATH_INFO: `/v3/users/17881e2146e244528dc404950291f6c2` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:46.967856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] RBAC: Authorizing `identity:delete_user(user_id=17881e2146e244528dc404950291f6c2)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:46.969936 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:47.025439 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17881e2146e244528dc404950291f6c2'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.026363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17881e2146e244528dc404950291f6c2'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.041527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17881e2146e244528dc404950291f6c2'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.042272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17881e2146e244528dc404950291f6c2'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.049437 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17881e2146e244528dc404950291f6c2'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.063615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1055/2114] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:46 2022] DELETE /identity/v3/users/17881e2146e244528dc404950291f6c2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:58:47.309184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:47.310882 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:47.332722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632aaa5-9f85-4517-957d-f5f6b8ceb902 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:47.359492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:47.360791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:47.427800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:47.428975 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:47.429252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:47.429523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] PATH_INFO: `/v3/users/8ef0eaa76ffe4b62a132f0f4ee595a6f` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:47.431079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef0eaa76ffe4b62a132f0f4ee595a6f)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:47.432973 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:47.470872 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14474396-dc4d-40e8-a39a-a8537cce312d admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:47.482695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:47.483631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:47.483911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:47.484363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] PATH_INFO: `/v3/projects/ee39471205eb4f1c83e7a34474c5fe06` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:47.485844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee39471205eb4f1c83e7a34474c5fe06)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:47.488309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:47.493078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef0eaa76ffe4b62a132f0f4ee595a6f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.512878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef0eaa76ffe4b62a132f0f4ee595a6f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.523173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef0eaa76ffe4b62a132f0f4ee595a6f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.526571 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef0eaa76ffe4b62a132f0f4ee595a6f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.533997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-9eeaacc8-9756-4eb3-abdb-e94846cbc4b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef0eaa76ffe4b62a132f0f4ee595a6f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.545285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1060/2115] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:47 2022] DELETE /identity/v3/users/8ef0eaa76ffe4b62a132f0f4ee595a6f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:47.546469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee39471205eb4f1c83e7a34474c5fe06', 'request_id': 'req-8a77c540-4ca8-4482-bebf-9001a9df2522'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.554463 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee39471205eb4f1c83e7a34474c5fe06', 'request_id': 'req-8a77c540-4ca8-4482-bebf-9001a9df2522'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.560497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee39471205eb4f1c83e7a34474c5fe06', 'request_id': 'req-8a77c540-4ca8-4482-bebf-9001a9df2522'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.575751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] The token cache is being invalidate because project ee39471205eb4f1c83e7a34474c5fe06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:47.576037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.576669 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8a77c540-4ca8-4482-bebf-9001a9df2522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.578235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1056/2116] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:47 2022] DELETE /identity/v3/projects/ee39471205eb4f1c83e7a34474c5fe06 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:47.583546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69986f-515c-4b80-b026-23cca9683ef3 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:47.583830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69986f-515c-4b80-b026-23cca9683ef3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:47.584106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69986f-515c-4b80-b026-23cca9683ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:47.603411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-3c69986f-515c-4b80-b026-23cca9683ef3 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:47.605505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-3c69986f-515c-4b80-b026-23cca9683ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:47.669943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1061/2117] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:47.678437 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0d350-92bb-4ea7-b150-7f7b93322c30 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:47.678741 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0d350-92bb-4ea7-b150-7f7b93322c30 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:47.679206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0d350-92bb-4ea7-b150-7f7b93322c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:47.699163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e3f0d350-92bb-4ea7-b150-7f7b93322c30 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:47.700441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e3f0d350-92bb-4ea7-b150-7f7b93322c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:47.709686 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1057/2118] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:47.714767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69986f-515c-4b80-b026-23cca9683ef3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:47.724743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69986f-515c-4b80-b026-23cca9683ef3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:47.730431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc30db-fd89-41c3-bd23-0b602273faae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:47.731326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc30db-fd89-41c3-bd23-0b602273faae admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:47.731763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc30db-fd89-41c3-bd23-0b602273faae admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:47.732093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc30db-fd89-41c3-bd23-0b602273faae admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:47.732997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc30db-fd89-41c3-bd23-0b602273faae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:58:47.733319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc30db-fd89-41c3-bd23-0b602273faae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:47.736599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc30db-fd89-41c3-bd23-0b602273faae admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:47.749763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1062/2119] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:58:47.750089 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0d350-92bb-4ea7-b150-7f7b93322c30 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:47.752274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e3f0d350-92bb-4ea7-b150-7f7b93322c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:47.753634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc30db-fd89-41c3-bd23-0b602273faae admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:47.767464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc30db-fd89-41c3-bd23-0b602273faae admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:47.773646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea75a31-72a2-49bf-bf05-80b86e281150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:47.774437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea75a31-72a2-49bf-bf05-80b86e281150 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:47.774980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea75a31-72a2-49bf-bf05-80b86e281150 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:47.775347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea75a31-72a2-49bf-bf05-80b86e281150 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:47.776301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea75a31-72a2-49bf-bf05-80b86e281150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:47.778975 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea75a31-72a2-49bf-bf05-80b86e281150 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:47.783222 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0d350-92bb-4ea7-b150-7f7b93322c30 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:47.798242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:47.799782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:47.800188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:47.800520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] PATH_INFO: `/v3/projects/703aa4774820401da920e420eb6181d9` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:47.802513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] RBAC: Authorizing `identity:delete_project(project_id=703aa4774820401da920e420eb6181d9)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:47.805778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:47.819983 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1063/2120] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:58:47 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:58:47.825912 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea75a31-72a2-49bf-bf05-80b86e281150 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:47.891162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703aa4774820401da920e420eb6181d9', 'request_id': 'req-4969deb5-1f9b-413e-bc3f-144a5225ccae'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.899081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703aa4774820401da920e420eb6181d9', 'request_id': 'req-4969deb5-1f9b-413e-bc3f-144a5225ccae'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.908277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea75a31-72a2-49bf-bf05-80b86e281150 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:47.908893 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703aa4774820401da920e420eb6181d9', 'request_id': 'req-4969deb5-1f9b-413e-bc3f-144a5225ccae'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.931942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] The token cache is being invalidate because project 703aa4774820401da920e420eb6181d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:47.932767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.934063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:47.936149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1058/2121] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:47 2022] DELETE /identity/v3/projects/703aa4774820401da920e420eb6181d9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:48.013050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d6feb-5d7c-44fc-b987-6247b296bc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.013733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d6feb-5d7c-44fc-b987-6247b296bc97 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.014017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d6feb-5d7c-44fc-b987-6247b296bc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.014298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d6feb-5d7c-44fc-b987-6247b296bc97 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.015058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d6feb-5d7c-44fc-b987-6247b296bc97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.017091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d6feb-5d7c-44fc-b987-6247b296bc97 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.040016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1064/2122] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:58:47 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:48.044536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.046185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:48.060815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969deb5-1f9b-413e-bc3f-144a5225ccae admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.065310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f1e62-beb9-4942-ab9f-7f03a65b9cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.066007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f1e62-beb9-4942-ab9f-7f03a65b9cbc admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.066344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f1e62-beb9-4942-ab9f-7f03a65b9cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.066566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f1e62-beb9-4942-ab9f-7f03a65b9cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.067365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f1e62-beb9-4942-ab9f-7f03a65b9cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.069037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f1e62-beb9-4942-ab9f-7f03a65b9cbc admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.081139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1059/2123] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:48.085182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d6feb-5d7c-44fc-b987-6247b296bc97 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.096050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d6feb-5d7c-44fc-b987-6247b296bc97 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.101740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ee18e-2486-4718-b680-f1f1dd5f7254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.103145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ee18e-2486-4718-b680-f1f1dd5f7254 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.103458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ee18e-2486-4718-b680-f1f1dd5f7254 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.103729 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ee18e-2486-4718-b680-f1f1dd5f7254 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.104505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ee18e-2486-4718-b680-f1f1dd5f7254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.106128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ee18e-2486-4718-b680-f1f1dd5f7254 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.120609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1065/2124] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:48.125984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f1e62-beb9-4942-ab9f-7f03a65b9cbc admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.136887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f1e62-beb9-4942-ab9f-7f03a65b9cbc admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.141832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146976be-0a44-4f41-8a81-6c7767b91605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.142910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146976be-0a44-4f41-8a81-6c7767b91605 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.143061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146976be-0a44-4f41-8a81-6c7767b91605 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.143275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146976be-0a44-4f41-8a81-6c7767b91605 admin admin] PATH_INFO: `/v3/projects/539151f2dc77400e97d1382c968ab449/users/52b14f82dac84f5c8bd61130b341eb75/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.163365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146976be-0a44-4f41-8a81-6c7767b91605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=539151f2dc77400e97d1382c968ab449, user_id=52b14f82dac84f5c8bd61130b341eb75, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.168298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146976be-0a44-4f41-8a81-6c7767b91605 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.183997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1060/2125] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:48 2022] PUT /identity/v3/projects/539151f2dc77400e97d1382c968ab449/users/52b14f82dac84f5c8bd61130b341eb75/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:48.195294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe755b1-e758-4ba0-b8ec-634d24c32fe2 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.195584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe755b1-e758-4ba0-b8ec-634d24c32fe2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.195891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe755b1-e758-4ba0-b8ec-634d24c32fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.222020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6fe755b1-e758-4ba0-b8ec-634d24c32fe2 None None] MFA Rules not processed for user `52b14f82dac84f5c8bd61130b341eb75`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:48.223267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6fe755b1-e758-4ba0-b8ec-634d24c32fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:48.287906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1066/2126] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:48.293093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146976be-0a44-4f41-8a81-6c7767b91605 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.355881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146976be-0a44-4f41-8a81-6c7767b91605 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.361925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a38078-270a-48b3-b5ca-dbbe314bcfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.362461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a38078-270a-48b3-b5ca-dbbe314bcfce admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.362769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a38078-270a-48b3-b5ca-dbbe314bcfce admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.363047 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a38078-270a-48b3-b5ca-dbbe314bcfce admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.363746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a38078-270a-48b3-b5ca-dbbe314bcfce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.365651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a38078-270a-48b3-b5ca-dbbe314bcfce admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.394665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1061/2127] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:58:48 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:48.398475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe755b1-e758-4ba0-b8ec-634d24c32fe2 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.412289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe755b1-e758-4ba0-b8ec-634d24c32fe2 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.494876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8527e1-d1c6-4428-b63f-abac4bbb5293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.495570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8527e1-d1c6-4428-b63f-abac4bbb5293 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.495853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8527e1-d1c6-4428-b63f-abac4bbb5293 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.496129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8527e1-d1c6-4428-b63f-abac4bbb5293 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.496920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8527e1-d1c6-4428-b63f-abac4bbb5293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.499618 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8527e1-d1c6-4428-b63f-abac4bbb5293 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.522626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1067/2128] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:58:48 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:48.526649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a38078-270a-48b3-b5ca-dbbe314bcfce admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.537190 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a38078-270a-48b3-b5ca-dbbe314bcfce admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.547433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57088f-a5e1-493a-b851-fd84be9188db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.548136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57088f-a5e1-493a-b851-fd84be9188db admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.548419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57088f-a5e1-493a-b851-fd84be9188db admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.548696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57088f-a5e1-493a-b851-fd84be9188db admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.549420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57088f-a5e1-493a-b851-fd84be9188db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.551154 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57088f-a5e1-493a-b851-fd84be9188db admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.562349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1062/2129] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:58:48.566402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8527e1-d1c6-4428-b63f-abac4bbb5293 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.576445 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8527e1-d1c6-4428-b63f-abac4bbb5293 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.581084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae0ce3-4d50-445f-9271-e19bd664d8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.581739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae0ce3-4d50-445f-9271-e19bd664d8fa admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.582023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae0ce3-4d50-445f-9271-e19bd664d8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.582302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae0ce3-4d50-445f-9271-e19bd664d8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.583063 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae0ce3-4d50-445f-9271-e19bd664d8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.584656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae0ce3-4d50-445f-9271-e19bd664d8fa admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.595721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1068/2130] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:48.599865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57088f-a5e1-493a-b851-fd84be9188db admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.610579 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57088f-a5e1-493a-b851-fd84be9188db admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.616412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80093b0-5712-4451-8219-81a7cccf2eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.617387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80093b0-5712-4451-8219-81a7cccf2eca admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.617695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80093b0-5712-4451-8219-81a7cccf2eca admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.617968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80093b0-5712-4451-8219-81a7cccf2eca admin admin] PATH_INFO: `/v3/projects/e6c33a8bb2f9431fa8ba2985732ef03e/users/9bb8dedda89548759516941c1efbf94c/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.632659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80093b0-5712-4451-8219-81a7cccf2eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c33a8bb2f9431fa8ba2985732ef03e, user_id=9bb8dedda89548759516941c1efbf94c, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.635383 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80093b0-5712-4451-8219-81a7cccf2eca admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.648533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1063/2131] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:48 2022] PUT /identity/v3/projects/e6c33a8bb2f9431fa8ba2985732ef03e/users/9bb8dedda89548759516941c1efbf94c/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:48.659210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86df24b-eef0-4c2b-aaf0-01ee8152fc4e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.659431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86df24b-eef0-4c2b-aaf0-01ee8152fc4e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.659707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86df24b-eef0-4c2b-aaf0-01ee8152fc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.687339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a86df24b-eef0-4c2b-aaf0-01ee8152fc4e None None] MFA Rules not processed for user `9bb8dedda89548759516941c1efbf94c`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:48.688624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a86df24b-eef0-4c2b-aaf0-01ee8152fc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:48.753800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1069/2132] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:48.758266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80093b0-5712-4451-8219-81a7cccf2eca admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.819617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80093b0-5712-4451-8219-81a7cccf2eca admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.824468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fa5df-4dfe-4367-87f4-624764296281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.825115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fa5df-4dfe-4367-87f4-624764296281 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.825396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fa5df-4dfe-4367-87f4-624764296281 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.825673 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fa5df-4dfe-4367-87f4-624764296281 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.826357 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fa5df-4dfe-4367-87f4-624764296281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.828280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fa5df-4dfe-4367-87f4-624764296281 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.853879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1064/2133] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:58:48 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:48.857921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86df24b-eef0-4c2b-aaf0-01ee8152fc4e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.868440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86df24b-eef0-4c2b-aaf0-01ee8152fc4e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.925462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded09987-590e-40d7-abea-6e13e435ff49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.926038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded09987-590e-40d7-abea-6e13e435ff49 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.926315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded09987-590e-40d7-abea-6e13e435ff49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.926584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded09987-590e-40d7-abea-6e13e435ff49 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.927323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded09987-590e-40d7-abea-6e13e435ff49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.929516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded09987-590e-40d7-abea-6e13e435ff49 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.950278 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1070/2134] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:58:48 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:48.954329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fa5df-4dfe-4367-87f4-624764296281 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.965165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fa5df-4dfe-4367-87f4-624764296281 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:48.970065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a6576-f1d8-4300-bbf3-ceb3eb094586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:48.970792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a6576-f1d8-4300-bbf3-ceb3eb094586 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:48.971029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a6576-f1d8-4300-bbf3-ceb3eb094586 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:48.971309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a6576-f1d8-4300-bbf3-ceb3eb094586 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:48.972041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a6576-f1d8-4300-bbf3-ceb3eb094586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:48.973642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a6576-f1d8-4300-bbf3-ceb3eb094586 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:48.984351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1065/2135] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:48.988661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded09987-590e-40d7-abea-6e13e435ff49 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:48.998725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded09987-590e-40d7-abea-6e13e435ff49 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:49.003660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8780c-de8c-4173-a153-67e6c8555679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:49.004657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8780c-de8c-4173-a153-67e6c8555679 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:49.004944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8780c-de8c-4173-a153-67e6c8555679 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:49.005364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8780c-de8c-4173-a153-67e6c8555679 admin admin] PATH_INFO: `/v3/projects/92914e654abf4243b7407635009616b1/users/07882097c438471d86eb35f2ec79584f/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:49.021835 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8780c-de8c-4173-a153-67e6c8555679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92914e654abf4243b7407635009616b1, user_id=07882097c438471d86eb35f2ec79584f, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:49.025189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8780c-de8c-4173-a153-67e6c8555679 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:49.038894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1071/2136] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:48 2022] PUT /identity/v3/projects/92914e654abf4243b7407635009616b1/users/07882097c438471d86eb35f2ec79584f/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:49.043055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a6576-f1d8-4300-bbf3-ceb3eb094586 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:49.053381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a6576-f1d8-4300-bbf3-ceb3eb094586 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:49.109385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a363a-7904-456e-8def-e83a7843d3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:49.110117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a363a-7904-456e-8def-e83a7843d3da admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:49.110404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a363a-7904-456e-8def-e83a7843d3da admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:49.110674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a363a-7904-456e-8def-e83a7843d3da admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:49.111429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a363a-7904-456e-8def-e83a7843d3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:49.113087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a363a-7904-456e-8def-e83a7843d3da admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:49.123283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1066/2137] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:49.127464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8780c-de8c-4173-a153-67e6c8555679 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:49.138466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8780c-de8c-4173-a153-67e6c8555679 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:49.144027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccd635-5c9f-4be5-a1f0-d5f4b80c6df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:49.144972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccd635-5c9f-4be5-a1f0-d5f4b80c6df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:49.145315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccd635-5c9f-4be5-a1f0-d5f4b80c6df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:49.145651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccd635-5c9f-4be5-a1f0-d5f4b80c6df2 admin admin] PATH_INFO: `/v3/projects/92914e654abf4243b7407635009616b1/users/07882097c438471d86eb35f2ec79584f/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:49.147702 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccd635-5c9f-4be5-a1f0-d5f4b80c6df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92914e654abf4243b7407635009616b1, user_id=07882097c438471d86eb35f2ec79584f, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:49.150987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccd635-5c9f-4be5-a1f0-d5f4b80c6df2 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:49.161856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1072/2138] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:49 2022] PUT /identity/v3/projects/92914e654abf4243b7407635009616b1/users/07882097c438471d86eb35f2ec79584f/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:49.173172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891b797-7db5-4bbb-a406-853536235f48 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:49.173475 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891b797-7db5-4bbb-a406-853536235f48 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:49.173757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891b797-7db5-4bbb-a406-853536235f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:49.201461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e891b797-7db5-4bbb-a406-853536235f48 None None] MFA Rules not processed for user `07882097c438471d86eb35f2ec79584f`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:49.202702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e891b797-7db5-4bbb-a406-853536235f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:49.274725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1067/2139] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:49.286124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccd635-5c9f-4be5-a1f0-d5f4b80c6df2 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:49.287916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a1ccd635-5c9f-4be5-a1f0-d5f4b80c6df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:49.343400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccd635-5c9f-4be5-a1f0-d5f4b80c6df2 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:49.347158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4777f-1d41-41ac-bbbf-ce614d749a1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:49.347799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4777f-1d41-41ac-bbbf-ce614d749a1d service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:49.348080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4777f-1d41-41ac-bbbf-ce614d749a1d service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:49.348348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4777f-1d41-41ac-bbbf-ce614d749a1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:49.401816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4777f-1d41-41ac-bbbf-ce614d749a1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:49.403294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4777f-1d41-41ac-bbbf-ce614d749a1d service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:49.412444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1073/2140] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:50.929808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891b797-7db5-4bbb-a406-853536235f48 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:50.932790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e891b797-7db5-4bbb-a406-853536235f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:50.948386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891b797-7db5-4bbb-a406-853536235f48 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:50.952936 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1cb7f-a77d-48f9-abcc-66883e77df2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:50.953844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1cb7f-a77d-48f9-abcc-66883e77df2e service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:50.954017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1cb7f-a77d-48f9-abcc-66883e77df2e service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:50.954298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1cb7f-a77d-48f9-abcc-66883e77df2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:51.015648 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1cb7f-a77d-48f9-abcc-66883e77df2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:51.017264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1cb7f-a77d-48f9-abcc-66883e77df2e service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:51.026983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1068/2141] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:51.588814 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4777f-1d41-41ac-bbbf-ce614d749a1d service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:51.597300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4777f-1d41-41ac-bbbf-ce614d749a1d service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:51.600971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79973ca-d242-407c-905a-ebbe50211d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:51.601610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79973ca-d242-407c-905a-ebbe50211d78 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:51.606569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79973ca-d242-407c-905a-ebbe50211d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:51.606569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79973ca-d242-407c-905a-ebbe50211d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:51.613239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79973ca-d242-407c-905a-ebbe50211d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:51.614669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79973ca-d242-407c-905a-ebbe50211d78 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:51.625038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1074/2142] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:58:55.574004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1cb7f-a77d-48f9-abcc-66883e77df2e service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:55.635425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1cb7f-a77d-48f9-abcc-66883e77df2e service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:55.640144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:55.641011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:55.641292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:55.641565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] PATH_INFO: `/v3/users/52b14f82dac84f5c8bd61130b341eb75` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:55.642785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52b14f82dac84f5c8bd61130b341eb75)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:55.644838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:55.694183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52b14f82dac84f5c8bd61130b341eb75'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:55.694911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52b14f82dac84f5c8bd61130b341eb75'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:55.706973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52b14f82dac84f5c8bd61130b341eb75'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:55.707903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52b14f82dac84f5c8bd61130b341eb75'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:55.720781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52b14f82dac84f5c8bd61130b341eb75'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:55.735279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1069/2143] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:55 2022] DELETE /identity/v3/users/52b14f82dac84f5c8bd61130b341eb75 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:55.739988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b02ca-c1b3-4f80-a83a-aa688efaddf7 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:55.740533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b02ca-c1b3-4f80-a83a-aa688efaddf7 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:55.740627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b02ca-c1b3-4f80-a83a-aa688efaddf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:55.758816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-975b02ca-c1b3-4f80-a83a-aa688efaddf7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:55.760434 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-975b02ca-c1b3-4f80-a83a-aa688efaddf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:55.822095 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1075/2144] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:55.832132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:55.833723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:55.901151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff622ef-cbbc-4dfd-9286-03a23f68db54 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:55.904927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293ac72-64e8-4238-9bd1-79933a822131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:55.905576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ac72-64e8-4238-9bd1-79933a822131 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:55.905857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ac72-64e8-4238-9bd1-79933a822131 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:55.906132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293ac72-64e8-4238-9bd1-79933a822131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:55.920927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293ac72-64e8-4238-9bd1-79933a822131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:55.922375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293ac72-64e8-4238-9bd1-79933a822131 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:55.933211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1070/2145] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:58:56.235577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b02ca-c1b3-4f80-a83a-aa688efaddf7 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:56.237496 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-975b02ca-c1b3-4f80-a83a-aa688efaddf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:56.251602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b02ca-c1b3-4f80-a83a-aa688efaddf7 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:56.257202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:56.263069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:56.263399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:56.263681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] PATH_INFO: `/v3/projects/539151f2dc77400e97d1382c968ab449` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:56.265290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=539151f2dc77400e97d1382c968ab449)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:56.267474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:56.330424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '539151f2dc77400e97d1382c968ab449', 'request_id': 'req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.336697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '539151f2dc77400e97d1382c968ab449', 'request_id': 'req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.345713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '539151f2dc77400e97d1382c968ab449', 'request_id': 'req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.358753 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] The token cache is being invalidate because project 539151f2dc77400e97d1382c968ab449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:56.358993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.359666 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.361381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1076/2146] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:56 2022] DELETE /identity/v3/projects/539151f2dc77400e97d1382c968ab449 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:56.365340 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293ac72-64e8-4238-9bd1-79933a822131 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:56.379003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293ac72-64e8-4238-9bd1-79933a822131 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:56.441648 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:56.442530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:56.442836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:56.443110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] PATH_INFO: `/v3/users/9bb8dedda89548759516941c1efbf94c` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:56.444335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb8dedda89548759516941c1efbf94c)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:56.447278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:56.497978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb8dedda89548759516941c1efbf94c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.498696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb8dedda89548759516941c1efbf94c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.511235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb8dedda89548759516941c1efbf94c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.511930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb8dedda89548759516941c1efbf94c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.519291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb8dedda89548759516941c1efbf94c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.534630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1071/2147] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:56 2022] DELETE /identity/v3/users/9bb8dedda89548759516941c1efbf94c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:56.810030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:56.812072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:56.842068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9bdb2-5bd5-45d2-aa78-6fe9bb060155 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:56.899002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:56.899765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:56.900044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:56.900316 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] PATH_INFO: `/v3/projects/e6c33a8bb2f9431fa8ba2985732ef03e` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:56.901508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c33a8bb2f9431fa8ba2985732ef03e)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:56.903764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:56.954195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c33a8bb2f9431fa8ba2985732ef03e', 'request_id': 'req-b4d66a75-634b-44a8-b78c-db79bc6fc6de'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.959329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c33a8bb2f9431fa8ba2985732ef03e', 'request_id': 'req-b4d66a75-634b-44a8-b78c-db79bc6fc6de'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.967104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c33a8bb2f9431fa8ba2985732ef03e', 'request_id': 'req-b4d66a75-634b-44a8-b78c-db79bc6fc6de'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.979392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] The token cache is being invalidate because project e6c33a8bb2f9431fa8ba2985732ef03e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:56.979681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.980331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4d66a75-634b-44a8-b78c-db79bc6fc6de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:56.981886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1077/2148] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:56 2022] DELETE /identity/v3/projects/e6c33a8bb2f9431fa8ba2985732ef03e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:56.985778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:56.987266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:57.000370 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d446eaf-ab4d-4403-aaeb-5291440cb924 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:57.069364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:57.071260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:57.071656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:57.071938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] PATH_INFO: `/v3/users/07882097c438471d86eb35f2ec79584f` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:57.073215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=07882097c438471d86eb35f2ec79584f)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:57.073513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c04a0-3d67-464f-aedd-1c09847f292d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:57.073810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c04a0-3d67-464f-aedd-1c09847f292d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:57.074088 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c04a0-3d67-464f-aedd-1c09847f292d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:57.075265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:57.092741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-173c04a0-3d67-464f-aedd-1c09847f292d None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:57.094085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-173c04a0-3d67-464f-aedd-1c09847f292d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:57.125971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07882097c438471d86eb35f2ec79584f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:57.126736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07882097c438471d86eb35f2ec79584f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:57.138308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07882097c438471d86eb35f2ec79584f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:57.138999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07882097c438471d86eb35f2ec79584f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:57.145694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07882097c438471d86eb35f2ec79584f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:57.151541 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1078/2149] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 13:58:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:58:57.159863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1072/2150] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:58:56 2022] DELETE /identity/v3/users/07882097c438471d86eb35f2ec79584f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:57.164786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c04a0-3d67-464f-aedd-1c09847f292d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:57.166197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-173c04a0-3d67-464f-aedd-1c09847f292d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:57.235662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c04a0-3d67-464f-aedd-1c09847f292d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:57.300148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8447935-cf0f-4afb-8908-5fd9de7e9c12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:57.300801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447935-cf0f-4afb-8908-5fd9de7e9c12 service swift] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:57.301078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447935-cf0f-4afb-8908-5fd9de7e9c12 service swift] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:57.301349 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447935-cf0f-4afb-8908-5fd9de7e9c12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:57.374677 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8447935-cf0f-4afb-8908-5fd9de7e9c12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:57.376480 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8447935-cf0f-4afb-8908-5fd9de7e9c12 service swift] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:57.386988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1079/2151] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 13:58:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:58:57.481379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:57.484578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:57.633431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73cf50-ab1b-42e9-9a82-94cdcd6a75ce admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:57.638683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:57.639723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:57.640009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:57.640275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] PATH_INFO: `/v3/projects/92914e654abf4243b7407635009616b1` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:57.641594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92914e654abf4243b7407635009616b1)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:57.643792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:57.706856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92914e654abf4243b7407635009616b1', 'request_id': 'req-90d564d3-2372-4e06-8a60-3ab7d9227101'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:57.714090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92914e654abf4243b7407635009616b1', 'request_id': 'req-90d564d3-2372-4e06-8a60-3ab7d9227101'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:57.721014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92914e654abf4243b7407635009616b1', 'request_id': 'req-90d564d3-2372-4e06-8a60-3ab7d9227101'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:57.736478 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] The token cache is being invalidate because project 92914e654abf4243b7407635009616b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:58:57.736793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:57.737590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90d564d3-2372-4e06-8a60-3ab7d9227101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:58:57.742758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1073/2152] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:58:57 2022] DELETE /identity/v3/projects/92914e654abf4243b7407635009616b1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:57.746950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e29d4a-088e-4f49-bc27-5876334b154a None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:57.747579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e29d4a-088e-4f49-bc27-5876334b154a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:57.748176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e29d4a-088e-4f49-bc27-5876334b154a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:57.771564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f7e29d4a-088e-4f49-bc27-5876334b154a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:57.773143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f7e29d4a-088e-4f49-bc27-5876334b154a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:57.852485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1080/2153] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:57.860006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4914e0c-af0a-4f91-829e-9476a48c9054 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:57.860470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4914e0c-af0a-4f91-829e-9476a48c9054 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:57.860839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4914e0c-af0a-4f91-829e-9476a48c9054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:57.885804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-d4914e0c-af0a-4f91-829e-9476a48c9054 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:57.887196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d4914e0c-af0a-4f91-829e-9476a48c9054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:57.900394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1074/2154] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:57.904556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e29d4a-088e-4f49-bc27-5876334b154a None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:57.922299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e29d4a-088e-4f49-bc27-5876334b154a None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:57.943173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ccf4a4-475b-465f-b1e5-e706749fd74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:57.943859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ccf4a4-475b-465f-b1e5-e706749fd74a admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:57.944138 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ccf4a4-475b-465f-b1e5-e706749fd74a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:57.944411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ccf4a4-475b-465f-b1e5-e706749fd74a admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:57.945675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ccf4a4-475b-465f-b1e5-e706749fd74a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:58:57.945988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ccf4a4-475b-465f-b1e5-e706749fd74a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:57.947623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ccf4a4-475b-465f-b1e5-e706749fd74a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:57.964329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1081/2155] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:58:57.968199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4914e0c-af0a-4f91-829e-9476a48c9054 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:57.983102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4914e0c-af0a-4f91-829e-9476a48c9054 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:57.988623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7862b1b-43a3-4203-9e47-4d6d6b4e560d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:57.990391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7862b1b-43a3-4203-9e47-4d6d6b4e560d admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:57.990391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7862b1b-43a3-4203-9e47-4d6d6b4e560d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:57.990391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7862b1b-43a3-4203-9e47-4d6d6b4e560d admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:57.990391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7862b1b-43a3-4203-9e47-4d6d6b4e560d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:57.992512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7862b1b-43a3-4203-9e47-4d6d6b4e560d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:58.025849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1075/2156] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:58:57 2022] POST /identity/v3/projects => generated 397 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:58.031610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ccf4a4-475b-465f-b1e5-e706749fd74a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:58.043931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ccf4a4-475b-465f-b1e5-e706749fd74a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:58.131996 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3a454-6d55-47a6-bb69-f16d979a9426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:58.133207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3a454-6d55-47a6-bb69-f16d979a9426 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:58.133497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3a454-6d55-47a6-bb69-f16d979a9426 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:58.133775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3a454-6d55-47a6-bb69-f16d979a9426 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:58.134499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3a454-6d55-47a6-bb69-f16d979a9426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:58.136682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3a454-6d55-47a6-bb69-f16d979a9426 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:58.163914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1082/2157] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:58:58 2022] POST /identity/v3/users => generated 299 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:58:58.167787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7862b1b-43a3-4203-9e47-4d6d6b4e560d admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:58.180573 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7862b1b-43a3-4203-9e47-4d6d6b4e560d admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:58.185842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18566114-6a08-46fb-b024-bcc440b1b11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:58.186519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18566114-6a08-46fb-b024-bcc440b1b11f admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:58.186839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18566114-6a08-46fb-b024-bcc440b1b11f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:58.187120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18566114-6a08-46fb-b024-bcc440b1b11f admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:58.187848 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18566114-6a08-46fb-b024-bcc440b1b11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:58.189459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18566114-6a08-46fb-b024-bcc440b1b11f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:58.202750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1076/2158] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:58.206651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3a454-6d55-47a6-bb69-f16d979a9426 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:58.220200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3a454-6d55-47a6-bb69-f16d979a9426 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:58.225250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab821afd-3729-406f-8365-7b9738f5b86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:58.225928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab821afd-3729-406f-8365-7b9738f5b86b admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:58.226224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab821afd-3729-406f-8365-7b9738f5b86b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:58.226560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab821afd-3729-406f-8365-7b9738f5b86b admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:58.227356 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab821afd-3729-406f-8365-7b9738f5b86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:58.229149 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab821afd-3729-406f-8365-7b9738f5b86b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:58.240260 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1083/2159] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:58:58.246920 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18566114-6a08-46fb-b024-bcc440b1b11f admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:58.261046 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18566114-6a08-46fb-b024-bcc440b1b11f admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:58.269151 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712ad32-cbf1-458c-bd18-5dbea6030a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:58.270045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712ad32-cbf1-458c-bd18-5dbea6030a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:58.270320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712ad32-cbf1-458c-bd18-5dbea6030a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:58.270591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712ad32-cbf1-458c-bd18-5dbea6030a6e admin admin] PATH_INFO: `/v3/projects/6faceecb7e444ec1926eff01ab6663eb/users/22df6f1063ab4f85a8032daedd7824f2/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:58.289424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712ad32-cbf1-458c-bd18-5dbea6030a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6faceecb7e444ec1926eff01ab6663eb, user_id=22df6f1063ab4f85a8032daedd7824f2, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:58.292185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712ad32-cbf1-458c-bd18-5dbea6030a6e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:58.311610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1077/2160] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:58:58 2022] PUT /identity/v3/projects/6faceecb7e444ec1926eff01ab6663eb/users/22df6f1063ab4f85a8032daedd7824f2/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:58:58.322926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44632-fba0-456f-9dae-d2af86f45c61 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:58.323225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44632-fba0-456f-9dae-d2af86f45c61 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:58.323532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44632-fba0-456f-9dae-d2af86f45c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:58.354026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-aef44632-fba0-456f-9dae-d2af86f45c61 None None] MFA Rules not processed for user `22df6f1063ab4f85a8032daedd7824f2`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:58:58.355350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-aef44632-fba0-456f-9dae-d2af86f45c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:58.470888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1084/2161] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:58:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:58:58.482330 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712ad32-cbf1-458c-bd18-5dbea6030a6e admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:58:58.484301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5712ad32-cbf1-458c-bd18-5dbea6030a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:58:58.573512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712ad32-cbf1-458c-bd18-5dbea6030a6e admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:58:58.577235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946041c-3ec9-4d93-a364-101b18b2711b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:58:58.577888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946041c-3ec9-4d93-a364-101b18b2711b service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:58:58.578166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946041c-3ec9-4d93-a364-101b18b2711b service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:58:58.578445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946041c-3ec9-4d93-a364-101b18b2711b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:58:58.595637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946041c-3ec9-4d93-a364-101b18b2711b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:58:58.597130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946041c-3ec9-4d93-a364-101b18b2711b service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:58:58.633719 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1078/2162] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:58:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:59:10.628853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef44632-fba0-456f-9dae-d2af86f45c61 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:10.723909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef44632-fba0-456f-9dae-d2af86f45c61 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:10.731509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:10.732442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:10.732719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:10.733073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] PATH_INFO: `/v3/users/22df6f1063ab4f85a8032daedd7824f2` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:10.734657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] RBAC: Authorizing `identity:delete_user(user_id=22df6f1063ab4f85a8032daedd7824f2)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:10.737654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:10.805193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22df6f1063ab4f85a8032daedd7824f2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:59:10.824619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22df6f1063ab4f85a8032daedd7824f2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:59:10.838264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22df6f1063ab4f85a8032daedd7824f2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:59:10.839417 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22df6f1063ab4f85a8032daedd7824f2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:59:10.848907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22df6f1063ab4f85a8032daedd7824f2'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:59:10.857545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1085/2163] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 13:59:10 2022] DELETE /identity/v3/users/22df6f1063ab4f85a8032daedd7824f2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:59:10.862428 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7002510-ff43-4585-8c4e-ad8fb72dfe0e None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:10.863062 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7002510-ff43-4585-8c4e-ad8fb72dfe0e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:10.863173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7002510-ff43-4585-8c4e-ad8fb72dfe0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:10.880999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a7002510-ff43-4585-8c4e-ad8fb72dfe0e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:10.882817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a7002510-ff43-4585-8c4e-ad8fb72dfe0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:10.956914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1079/2164] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:59:10.968172 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:10.969611 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:11.055904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f24676-1d5f-47c0-8acb-4d02b623a46a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:11.061324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf177761-7aac-42b7-b2a0-33a7add6f5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:11.062074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf177761-7aac-42b7-b2a0-33a7add6f5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:11.062349 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf177761-7aac-42b7-b2a0-33a7add6f5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:11.062738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf177761-7aac-42b7-b2a0-33a7add6f5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:11.074907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf177761-7aac-42b7-b2a0-33a7add6f5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:11.086232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf177761-7aac-42b7-b2a0-33a7add6f5d5 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:11.088336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1086/2165] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:59:11.752031 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7002510-ff43-4585-8c4e-ad8fb72dfe0e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:11.753574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a7002510-ff43-4585-8c4e-ad8fb72dfe0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:11.772193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7002510-ff43-4585-8c4e-ad8fb72dfe0e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:11.778067 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:11.779842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:11.780128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:11.780411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] PATH_INFO: `/v3/projects/6faceecb7e444ec1926eff01ab6663eb` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:11.782142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6faceecb7e444ec1926eff01ab6663eb)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:11.785050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:11.857107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6faceecb7e444ec1926eff01ab6663eb', 'request_id': 'req-f1514d6f-cb84-4c2f-9c35-67ece67436e5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:59:11.865029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6faceecb7e444ec1926eff01ab6663eb', 'request_id': 'req-f1514d6f-cb84-4c2f-9c35-67ece67436e5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:59:11.872399 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6faceecb7e444ec1926eff01ab6663eb', 'request_id': 'req-f1514d6f-cb84-4c2f-9c35-67ece67436e5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:59:11.895520 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] The token cache is being invalidate because project 6faceecb7e444ec1926eff01ab6663eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:59:11.895800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:59:11.896474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f1514d6f-cb84-4c2f-9c35-67ece67436e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:59:11.898104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1080/2166] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 13:59:11 2022] DELETE /identity/v3/projects/6faceecb7e444ec1926eff01ab6663eb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:59:11.907549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327168f-1c53-41f8-a1a2-a91e7b0ee770 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:11.908099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327168f-1c53-41f8-a1a2-a91e7b0ee770 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:11.908479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327168f-1c53-41f8-a1a2-a91e7b0ee770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:11.932656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0327168f-1c53-41f8-a1a2-a91e7b0ee770 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:11.933894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0327168f-1c53-41f8-a1a2-a91e7b0ee770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:12.014759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1087/2167] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:59:12.022516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd15647-d6a0-4af1-bb56-f35c54519fb2 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:12.023334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd15647-d6a0-4af1-bb56-f35c54519fb2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:12.023616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd15647-d6a0-4af1-bb56-f35c54519fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:12.044410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-bfd15647-d6a0-4af1-bb56-f35c54519fb2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:12.046358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-bfd15647-d6a0-4af1-bb56-f35c54519fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:12.057363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1081/2168] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:59:12.062146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327168f-1c53-41f8-a1a2-a91e7b0ee770 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:12.074140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327168f-1c53-41f8-a1a2-a91e7b0ee770 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:12.080203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52832933-3cdd-44ec-b61c-da0e56251d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:12.080881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52832933-3cdd-44ec-b61c-da0e56251d9a admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:12.081159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52832933-3cdd-44ec-b61c-da0e56251d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:12.081432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52832933-3cdd-44ec-b61c-da0e56251d9a admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:12.082179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52832933-3cdd-44ec-b61c-da0e56251d9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:59:12.082484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52832933-3cdd-44ec-b61c-da0e56251d9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:12.084157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52832933-3cdd-44ec-b61c-da0e56251d9a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:12.097914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1088/2169] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 13:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:59:12.103742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd15647-d6a0-4af1-bb56-f35c54519fb2 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:12.114105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd15647-d6a0-4af1-bb56-f35c54519fb2 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:12.118956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61c47a-5f8e-44ae-a952-90ef6b569603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:12.119627 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c47a-5f8e-44ae-a952-90ef6b569603 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:12.119904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c47a-5f8e-44ae-a952-90ef6b569603 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:12.120175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c47a-5f8e-44ae-a952-90ef6b569603 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:12.120863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61c47a-5f8e-44ae-a952-90ef6b569603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:12.122798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61c47a-5f8e-44ae-a952-90ef6b569603 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:12.157490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1082/2170] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:59:12 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:59:12.162315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52832933-3cdd-44ec-b61c-da0e56251d9a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:12.173369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52832933-3cdd-44ec-b61c-da0e56251d9a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:12.245060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d75c13-1695-4a25-9783-2d7abe1c3331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:12.245978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d75c13-1695-4a25-9783-2d7abe1c3331 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:12.246390 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d75c13-1695-4a25-9783-2d7abe1c3331 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:12.246961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d75c13-1695-4a25-9783-2d7abe1c3331 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:12.247656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d75c13-1695-4a25-9783-2d7abe1c3331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:12.249606 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d75c13-1695-4a25-9783-2d7abe1c3331 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:12.274018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1089/2171] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:59:12 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:59:12.278665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61c47a-5f8e-44ae-a952-90ef6b569603 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:12.292885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61c47a-5f8e-44ae-a952-90ef6b569603 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:12.298420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed11c55-797f-4205-af7c-4914c861b8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:12.299147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed11c55-797f-4205-af7c-4914c861b8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:12.299424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed11c55-797f-4205-af7c-4914c861b8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:12.299699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed11c55-797f-4205-af7c-4914c861b8c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:12.300433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed11c55-797f-4205-af7c-4914c861b8c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:12.302317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed11c55-797f-4205-af7c-4914c861b8c1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:12.328213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1083/2172] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:59:12.333188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d75c13-1695-4a25-9783-2d7abe1c3331 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:12.345052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d75c13-1695-4a25-9783-2d7abe1c3331 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:12.350064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf297822-f117-404d-b85d-405788bcc366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:12.350801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297822-f117-404d-b85d-405788bcc366 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:12.351089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297822-f117-404d-b85d-405788bcc366 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:12.351383 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297822-f117-404d-b85d-405788bcc366 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:12.352163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf297822-f117-404d-b85d-405788bcc366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:12.353924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf297822-f117-404d-b85d-405788bcc366 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:12.372364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1090/2173] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:59:12.377771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed11c55-797f-4205-af7c-4914c861b8c1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:12.390531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed11c55-797f-4205-af7c-4914c861b8c1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:12.396053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dcda82-e74f-4bdd-afef-903c96beb7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:12.396998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcda82-e74f-4bdd-afef-903c96beb7b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:12.397266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcda82-e74f-4bdd-afef-903c96beb7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:12.397565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcda82-e74f-4bdd-afef-903c96beb7b5 admin admin] PATH_INFO: `/v3/projects/fbbf18059349404c9ed7f53c5b38d2fd/users/a27e83f9f5fb4a0b99e7fac6ab90b3e4/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:12.418502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dcda82-e74f-4bdd-afef-903c96beb7b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbbf18059349404c9ed7f53c5b38d2fd, user_id=a27e83f9f5fb4a0b99e7fac6ab90b3e4, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:12.421266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dcda82-e74f-4bdd-afef-903c96beb7b5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:12.436853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1084/2174] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:59:12 2022] PUT /identity/v3/projects/fbbf18059349404c9ed7f53c5b38d2fd/users/a27e83f9f5fb4a0b99e7fac6ab90b3e4/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:59:12.442986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ede65-267a-4ce5-b951-f087b857a3d2 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:12.443506 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ede65-267a-4ce5-b951-f087b857a3d2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:12.444017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ede65-267a-4ce5-b951-f087b857a3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:12.468378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-412ede65-267a-4ce5-b951-f087b857a3d2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:12.469637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-412ede65-267a-4ce5-b951-f087b857a3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:12.563330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1091/2175] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:59:12.573550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dcda82-e74f-4bdd-afef-903c96beb7b5 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:12.575592 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d2dcda82-e74f-4bdd-afef-903c96beb7b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:12.680350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dcda82-e74f-4bdd-afef-903c96beb7b5 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:12.686345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02596f-3c94-46aa-a2d5-728ece56521f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:12.691317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02596f-3c94-46aa-a2d5-728ece56521f service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:12.691317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02596f-3c94-46aa-a2d5-728ece56521f service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:12.691317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02596f-3c94-46aa-a2d5-728ece56521f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:12.703713 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02596f-3c94-46aa-a2d5-728ece56521f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:12.705487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02596f-3c94-46aa-a2d5-728ece56521f service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:12.719074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1085/2176] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:59:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:59:20.855452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ede65-267a-4ce5-b951-f087b857a3d2 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:20.858425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-412ede65-267a-4ce5-b951-f087b857a3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:21.029236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ede65-267a-4ce5-b951-f087b857a3d2 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:21.037333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5c15f-1388-438a-93b6-12b4c10ca0a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ae9a6f1ac304e21b8d3115f2a0baf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:21.038859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5c15f-1388-438a-93b6-12b4c10ca0a5 service placement] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:21.039030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5c15f-1388-438a-93b6-12b4c10ca0a5 service placement] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:21.040004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5c15f-1388-438a-93b6-12b4c10ca0a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:21.042362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-49e5c15f-1388-438a-93b6-12b4c10ca0a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:21.064826 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5c15f-1388-438a-93b6-12b4c10ca0a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:21.066735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b68d5-7220-44ca-8599-c0b05db0cfb4 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:21.067226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b68d5-7220-44ca-8599-c0b05db0cfb4 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:21.067226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b68d5-7220-44ca-8599-c0b05db0cfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:21.083238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5c15f-1388-438a-93b6-12b4c10ca0a5 service placement] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:21.105052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1092/2177] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:59:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:59:21.117630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-6c8b68d5-7220-44ca-8599-c0b05db0cfb4 None None] MFA Rules not processed for user `a27e83f9f5fb4a0b99e7fac6ab90b3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:21.119624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6c8b68d5-7220-44ca-8599-c0b05db0cfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:21.250406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1086/2178] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:59:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:59:21.258131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5c15f-1388-438a-93b6-12b4c10ca0a5 service placement] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:21.271021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5c15f-1388-438a-93b6-12b4c10ca0a5 service placement] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:21.312677 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015c631-083f-4548-93e8-9cfb4714bc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:21.313829 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015c631-083f-4548-93e8-9cfb4714bc8d admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:21.314396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015c631-083f-4548-93e8-9cfb4714bc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:21.314988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015c631-083f-4548-93e8-9cfb4714bc8d admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:21.316177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015c631-083f-4548-93e8-9cfb4714bc8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:21.326344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015c631-083f-4548-93e8-9cfb4714bc8d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:21.442378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1093/2179] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 13:59:21 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:59:21.443801 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b68d5-7220-44ca-8599-c0b05db0cfb4 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:21.472491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b68d5-7220-44ca-8599-c0b05db0cfb4 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:21.593669 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa24338-0227-4f20-8b1a-790eb4599d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:21.594341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa24338-0227-4f20-8b1a-790eb4599d0f admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:21.597395 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa24338-0227-4f20-8b1a-790eb4599d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:21.597395 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa24338-0227-4f20-8b1a-790eb4599d0f admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:21.597395 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa24338-0227-4f20-8b1a-790eb4599d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:21.598112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa24338-0227-4f20-8b1a-790eb4599d0f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:21.630519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1087/2180] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 13:59:21 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:59:21.635257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015c631-083f-4548-93e8-9cfb4714bc8d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:21.653908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015c631-083f-4548-93e8-9cfb4714bc8d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:21.660016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cfe02-559b-4ac0-9eeb-32c244b4afbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:21.661152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cfe02-559b-4ac0-9eeb-32c244b4afbb admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:21.661648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cfe02-559b-4ac0-9eeb-32c244b4afbb admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:21.663131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cfe02-559b-4ac0-9eeb-32c244b4afbb admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:21.663577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546cfe02-559b-4ac0-9eeb-32c244b4afbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:21.668276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546cfe02-559b-4ac0-9eeb-32c244b4afbb admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:21.688135 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1094/2181] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:59:21.691007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa24338-0227-4f20-8b1a-790eb4599d0f admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:21.708906 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa24338-0227-4f20-8b1a-790eb4599d0f admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:21.718959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900afe38-3d54-4187-9ba6-151136a643bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:21.721521 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afe38-3d54-4187-9ba6-151136a643bf admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:21.721972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afe38-3d54-4187-9ba6-151136a643bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:21.722385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900afe38-3d54-4187-9ba6-151136a643bf admin admin] PATH_INFO: `/v3/projects/c54c962a109d4c52b606943ba49ade06/users/830ee6c1c2014a2892272854bad61457/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:21.749975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900afe38-3d54-4187-9ba6-151136a643bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54c962a109d4c52b606943ba49ade06, user_id=830ee6c1c2014a2892272854bad61457, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:21.755797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900afe38-3d54-4187-9ba6-151136a643bf admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:21.788808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1088/2182] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:59:21 2022] PUT /identity/v3/projects/c54c962a109d4c52b606943ba49ade06/users/830ee6c1c2014a2892272854bad61457/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:59:21.794249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cfe02-559b-4ac0-9eeb-32c244b4afbb admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:21.809995 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cfe02-559b-4ac0-9eeb-32c244b4afbb admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:21.946658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17a466-6e60-4827-827d-e62d42e84363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:21.948822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17a466-6e60-4827-827d-e62d42e84363 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:21.949255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17a466-6e60-4827-827d-e62d42e84363 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:21.949937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17a466-6e60-4827-827d-e62d42e84363 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:21.951308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17a466-6e60-4827-827d-e62d42e84363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:21.954142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17a466-6e60-4827-827d-e62d42e84363 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:21.992150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1095/2183] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 13:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:59:21.998871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900afe38-3d54-4187-9ba6-151136a643bf admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:22.026260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900afe38-3d54-4187-9ba6-151136a643bf admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:22.036441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd29ab-07b1-4e21-9c55-7eaf2e42f2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:22.037996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd29ab-07b1-4e21-9c55-7eaf2e42f2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:22.038480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd29ab-07b1-4e21-9c55-7eaf2e42f2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:22.039027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd29ab-07b1-4e21-9c55-7eaf2e42f2e0 admin admin] PATH_INFO: `/v3/projects/c54c962a109d4c52b606943ba49ade06/users/830ee6c1c2014a2892272854bad61457/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:22.043345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd29ab-07b1-4e21-9c55-7eaf2e42f2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54c962a109d4c52b606943ba49ade06, user_id=830ee6c1c2014a2892272854bad61457, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:22.051540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd29ab-07b1-4e21-9c55-7eaf2e42f2e0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:22.075115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1089/2184] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 13:59:21 2022] PUT /identity/v3/projects/c54c962a109d4c52b606943ba49ade06/users/830ee6c1c2014a2892272854bad61457/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:59:23.105448 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a25cd-4229-4797-961f-707c546bae05 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:23.106005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a25cd-4229-4797-961f-707c546bae05 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:23.106309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a25cd-4229-4797-961f-707c546bae05 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:23.108410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1096/2185] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 13:59:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:59:23.121386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf952b8-7c73-4ef2-b8c5-31db01bb49b5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:23.122206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf952b8-7c73-4ef2-b8c5-31db01bb49b5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:23.122206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf952b8-7c73-4ef2-b8c5-31db01bb49b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:23.260373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-0bf952b8-7c73-4ef2-b8c5-31db01bb49b5 None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:23.262949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0bf952b8-7c73-4ef2-b8c5-31db01bb49b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:23.486295 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1090/2186] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 13:59:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:59:23.509295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a25cd-4229-4797-961f-707c546bae05 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:23.513619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-4b3a25cd-4229-4797-961f-707c546bae05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:23.743072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a25cd-4229-4797-961f-707c546bae05 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:23.752346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aceb6e-de4d-4731-91d5-2201a48d7795 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4f45e859d84ceaa5a01477e83e098e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:23.755851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aceb6e-de4d-4731-91d5-2201a48d7795 service glance] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:23.756705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aceb6e-de4d-4731-91d5-2201a48d7795 service glance] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:23.757521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aceb6e-de4d-4731-91d5-2201a48d7795 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:23.795995 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aceb6e-de4d-4731-91d5-2201a48d7795 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:23.800175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aceb6e-de4d-4731-91d5-2201a48d7795 service glance] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:23.850952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1097/2187] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:59:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:59:23.976137 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7a608-2541-45d4-88d9-91e6fd01f8ac None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:23.978313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7a608-2541-45d4-88d9-91e6fd01f8ac None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:23.978929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7a608-2541-45d4-88d9-91e6fd01f8ac None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:23.982077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1091/2188] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 13:59:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:59:23.989377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25219d-f8e5-406e-a8d2-0db890e9c191 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:23.990355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25219d-f8e5-406e-a8d2-0db890e9c191 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:23.990899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25219d-f8e5-406e-a8d2-0db890e9c191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:24.018960 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-ea25219d-f8e5-406e-a8d2-0db890e9c191 None None] MFA Rules not processed for user `8f784cc461ab48ad92140680758a3704`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:24.024459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ea25219d-f8e5-406e-a8d2-0db890e9c191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:24.219371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1098/2189] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 13:59:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:59:24.238167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7a608-2541-45d4-88d9-91e6fd01f8ac None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:24.267689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7a608-2541-45d4-88d9-91e6fd01f8ac None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:24.271838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22465c24-f628-4d88-ba16-95a126b51992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:24.272675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22465c24-f628-4d88-ba16-95a126b51992 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:24.273059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22465c24-f628-4d88-ba16-95a126b51992 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:24.273438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22465c24-f628-4d88-ba16-95a126b51992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:24.282052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22465c24-f628-4d88-ba16-95a126b51992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:24.284353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22465c24-f628-4d88-ba16-95a126b51992 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:24.302554 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1092/2190] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:59:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:59:24.447866 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954d48-e058-4f04-9bfc-0fcec3d44d25 None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:24.448365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954d48-e058-4f04-9bfc-0fcec3d44d25 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:24.449056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954d48-e058-4f04-9bfc-0fcec3d44d25 None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:24.451802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1099/2191] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 13:59:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:59:24.468790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f3457-222c-4c45-a621-8bab2e3ce3fc None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:24.469375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f3457-222c-4c45-a621-8bab2e3ce3fc None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:24.469798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f3457-222c-4c45-a621-8bab2e3ce3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:24.519949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-906f3457-222c-4c45-a621-8bab2e3ce3fc None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:24.521599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-906f3457-222c-4c45-a621-8bab2e3ce3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:24.546635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1093/2192] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 13:59:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:59:24.561239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca954d48-e058-4f04-9bfc-0fcec3d44d25 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:24.584993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca954d48-e058-4f04-9bfc-0fcec3d44d25 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:24.598827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc833c-1dea-4f11-8808-539d354ecea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:24.600894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc833c-1dea-4f11-8808-539d354ecea0 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:24.605762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc833c-1dea-4f11-8808-539d354ecea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:24.606566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc833c-1dea-4f11-8808-539d354ecea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:24.649621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc833c-1dea-4f11-8808-539d354ecea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:24.655513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc833c-1dea-4f11-8808-539d354ecea0 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:24.700653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1100/2193] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:59:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:59:29.734124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c3efa-ea0b-49cf-9947-f6fe7bd52652 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:29.735202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c3efa-ea0b-49cf-9947-f6fe7bd52652 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:29.736824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c3efa-ea0b-49cf-9947-f6fe7bd52652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:29.789795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f83c3efa-ea0b-49cf-9947-f6fe7bd52652 None None] MFA Rules not processed for user `830ee6c1c2014a2892272854bad61457`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:29.792784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f83c3efa-ea0b-49cf-9947-f6fe7bd52652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:29.886488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1094/2194] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:59:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:59:29.904383 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc833c-1dea-4f11-8808-539d354ecea0 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:29.906407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-3dfc833c-1dea-4f11-8808-539d354ecea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:29.933015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc833c-1dea-4f11-8808-539d354ecea0 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:29.939559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a89fb-2196-402e-886e-35f8879273b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:29.940465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a89fb-2196-402e-886e-35f8879273b5 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:29.940905 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a89fb-2196-402e-886e-35f8879273b5 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:29.941433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a89fb-2196-402e-886e-35f8879273b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:30.052423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a89fb-2196-402e-886e-35f8879273b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:30.054649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a89fb-2196-402e-886e-35f8879273b5 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:30.086301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1101/2195] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 13:59:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:59:30.997965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3785c21-aea5-4be4-9f70-b2bbd29f3f5b None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:30.998865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3785c21-aea5-4be4-9f70-b2bbd29f3f5b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:30.998865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3785c21-aea5-4be4-9f70-b2bbd29f3f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:31.018178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-d3785c21-aea5-4be4-9f70-b2bbd29f3f5b None None] MFA Rules not processed for user `20fd230cc30f4980bedebc75c620d707`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:31.019775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d3785c21-aea5-4be4-9f70-b2bbd29f3f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:31.121506 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1095/2196] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:59:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:59:33.319345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27acda-5463-476f-b4b3-f25e935f326a None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:33.319862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27acda-5463-476f-b4b3-f25e935f326a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:33.320118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27acda-5463-476f-b4b3-f25e935f326a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:33.352463 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-7d27acda-5463-476f-b4b3-f25e935f326a None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:33.355128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7d27acda-5463-476f-b4b3-f25e935f326a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:33.527248 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1102/2197] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 13:59:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:59:33.537255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3785c21-aea5-4be4-9f70-b2bbd29f3f5b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:59:33.541029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d3785c21-aea5-4be4-9f70-b2bbd29f3f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:33.681372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3785c21-aea5-4be4-9f70-b2bbd29f3f5b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:59:33.687851 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e609e-bd4b-49f1-9d28-72ca96daa924 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:59:33.689230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e609e-bd4b-49f1-9d28-72ca96daa924 service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:33.689720 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e609e-bd4b-49f1-9d28-72ca96daa924 service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:33.690168 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e609e-bd4b-49f1-9d28-72ca96daa924 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:33.702461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e609e-bd4b-49f1-9d28-72ca96daa924 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:59:33.704639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e609e-bd4b-49f1-9d28-72ca96daa924 service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:59:33.726341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1096/2198] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 13:59:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:59:43.624952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2c6e-25b4-41fe-bc7f-6ac0da30b955 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:43.625628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2c6e-25b4-41fe-bc7f-6ac0da30b955 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:43.625628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2c6e-25b4-41fe-bc7f-6ac0da30b955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:43.645872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-feee2c6e-25b4-41fe-bc7f-6ac0da30b955 None None] MFA Rules not processed for user `902bea79f26a4c1d8c3c1937f99f9d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:43.647394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-feee2c6e-25b4-41fe-bc7f-6ac0da30b955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:43.738272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1103/2199] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:59:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:59:52.704119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357ebfa-08b0-47e0-a3ce-00666f93f019 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:59:52.704715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357ebfa-08b0-47e0-a3ce-00666f93f019 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:59:52.704715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357ebfa-08b0-47e0-a3ce-00666f93f019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:59:52.722933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-2357ebfa-08b0-47e0-a3ce-00666f93f019 None None] MFA Rules not processed for user `a27e83f9f5fb4a0b99e7fac6ab90b3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:59:52.724621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2357ebfa-08b0-47e0-a3ce-00666f93f019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:59:52.731958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1097/2200] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 13:59:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:00:02.846163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25684879-f432-4797-82c4-e37eb36666c1 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:00:02.847012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25684879-f432-4797-82c4-e37eb36666c1 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:00:02.847012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25684879-f432-4797-82c4-e37eb36666c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:00:02.868800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-25684879-f432-4797-82c4-e37eb36666c1 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:00:02.870331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-25684879-f432-4797-82c4-e37eb36666c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:00:02.882518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1104/2201] 10.222.0.79 () {58 vars in 967 bytes} [Fri May 13 14:00:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:00:02.896550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357ebfa-08b0-47e0-a3ce-00666f93f019 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:00:02.909004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357ebfa-08b0-47e0-a3ce-00666f93f019 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:00:02.920053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1d44c-5496-4694-b570-33a43ff78a3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:00:02.920847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1d44c-5496-4694-b570-33a43ff78a3a service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:00:02.921130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1d44c-5496-4694-b570-33a43ff78a3a service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:00:02.921427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1d44c-5496-4694-b570-33a43ff78a3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:00:02.934059 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1d44c-5496-4694-b570-33a43ff78a3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:00:02.937874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1d44c-5496-4694-b570-33a43ff78a3a service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:00:02.952082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1098/2202] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:00:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 14:00:25.140548 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25684879-f432-4797-82c4-e37eb36666c1 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:00:25.142346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-25684879-f432-4797-82c4-e37eb36666c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:00:25.161591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25684879-f432-4797-82c4-e37eb36666c1 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:00:25.166931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe54316-8a94-4e5d-bb43-50a8ee13a9b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:00:25.167917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe54316-8a94-4e5d-bb43-50a8ee13a9b4 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:00:25.168201 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe54316-8a94-4e5d-bb43-50a8ee13a9b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:00:25.168848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe54316-8a94-4e5d-bb43-50a8ee13a9b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:00:25.181032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe54316-8a94-4e5d-bb43-50a8ee13a9b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:00:25.183332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe54316-8a94-4e5d-bb43-50a8ee13a9b4 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:00:25.197393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1105/2203] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:00:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:00:28.872652 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa67919-f069-40db-ba85-1dd87aaf1576 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:00:28.873500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa67919-f069-40db-ba85-1dd87aaf1576 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:00:28.873500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa67919-f069-40db-ba85-1dd87aaf1576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:00:28.893716 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-baa67919-f069-40db-ba85-1dd87aaf1576 None None] MFA Rules not processed for user `648ae92126944b50a22ad780e92df914`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:00:28.894984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-baa67919-f069-40db-ba85-1dd87aaf1576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:00:28.980679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1099/2204] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:00:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:00:46.277807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe54316-8a94-4e5d-bb43-50a8ee13a9b4 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:00:46.292252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe54316-8a94-4e5d-bb43-50a8ee13a9b4 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:00:46.296598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a860f-185e-40bb-aac2-de62a1d0cf1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:00:46.297245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a860f-185e-40bb-aac2-de62a1d0cf1b service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:00:46.297527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a860f-185e-40bb-aac2-de62a1d0cf1b service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:00:46.297801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a860f-185e-40bb-aac2-de62a1d0cf1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:00:46.300284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7f0a860f-185e-40bb-aac2-de62a1d0cf1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:00:46.306273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa67919-f069-40db-ba85-1dd87aaf1576 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:00:46.316756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa67919-f069-40db-ba85-1dd87aaf1576 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:00:46.319202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a860f-185e-40bb-aac2-de62a1d0cf1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:00:46.320650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a860f-185e-40bb-aac2-de62a1d0cf1b service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:00:46.320940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a3d5f-43e9-4413-9d72-52de1eacf8a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:00:46.321723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a3d5f-43e9-4413-9d72-52de1eacf8a3 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:00:46.322012 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a3d5f-43e9-4413-9d72-52de1eacf8a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:00:46.322310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a3d5f-43e9-4413-9d72-52de1eacf8a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:00:46.324252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-561a3d5f-43e9-4413-9d72-52de1eacf8a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:00:46.333780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1106/2205] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:00:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:00:46.339823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a3d5f-43e9-4413-9d72-52de1eacf8a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:00:46.341261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a3d5f-43e9-4413-9d72-52de1eacf8a3 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:00:46.353112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1100/2206] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:00:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:00:56.525681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a860f-185e-40bb-aac2-de62a1d0cf1b service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:00:56.527322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7f0a860f-185e-40bb-aac2-de62a1d0cf1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:00:56.594343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a860f-185e-40bb-aac2-de62a1d0cf1b service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:00:56.599795 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:00:56.600658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:00:56.600930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:00:56.601199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] PATH_INFO: `/v3/users/902bea79f26a4c1d8c3c1937f99f9d4b` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:00:56.602379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=902bea79f26a4c1d8c3c1937f99f9d4b)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:00:56.604331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:00:56.651369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902bea79f26a4c1d8c3c1937f99f9d4b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:00:56.663464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902bea79f26a4c1d8c3c1937f99f9d4b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:00:56.670291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902bea79f26a4c1d8c3c1937f99f9d4b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:00:56.671028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902bea79f26a4c1d8c3c1937f99f9d4b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:00:56.678728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a7e52d11-617a-4983-9247-a0882f7b83b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902bea79f26a4c1d8c3c1937f99f9d4b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:00:56.685220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1107/2207] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:00:56 2022] DELETE /identity/v3/users/902bea79f26a4c1d8c3c1937f99f9d4b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:00:56.866946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a3d5f-43e9-4413-9d72-52de1eacf8a3 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:00:56.880578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a3d5f-43e9-4413-9d72-52de1eacf8a3 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:00:56.940983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:00:56.942508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:00:56.943028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:00:56.943254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] PATH_INFO: `/v3/projects/a6843d24fbf2449cbd117b8d9cfc7072` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:00:56.944542 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6843d24fbf2449cbd117b8d9cfc7072)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:00:56.946498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:00:56.991485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6843d24fbf2449cbd117b8d9cfc7072', 'request_id': 'req-27a0a056-328a-42ac-8655-f71f2913f788'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:00:56.997514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6843d24fbf2449cbd117b8d9cfc7072', 'request_id': 'req-27a0a056-328a-42ac-8655-f71f2913f788'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:00:57.002772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6843d24fbf2449cbd117b8d9cfc7072', 'request_id': 'req-27a0a056-328a-42ac-8655-f71f2913f788'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:00:57.015455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] The token cache is being invalidate because project a6843d24fbf2449cbd117b8d9cfc7072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:00:57.015766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:00:57.016413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:00:57.018054 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1101/2208] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:00:56 2022] DELETE /identity/v3/projects/a6843d24fbf2449cbd117b8d9cfc7072 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:01:14.491943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57b242-0566-4852-a272-723928c55c18 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:14.492858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57b242-0566-4852-a272-723928c55c18 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:14.492858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57b242-0566-4852-a272-723928c55c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:14.524160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-8e57b242-0566-4852-a272-723928c55c18 None None] MFA Rules not processed for user `8b2980196e0f4fafa5131dd5de02ac75`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:01:14.526375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-8e57b242-0566-4852-a272-723928c55c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:14.614185 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1108/2209] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:01:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:01:15.025532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:01:15.027840 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:15.100619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0a056-328a-42ac-8655-f71f2913f788 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:01:15.107129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba341138-f41c-41d5-82b4-756607911180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:01:15.108256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341138-f41c-41d5-82b4-756607911180 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:15.108762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341138-f41c-41d5-82b4-756607911180 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:15.109276 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba341138-f41c-41d5-82b4-756607911180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:15.113194 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ba341138-f41c-41d5-82b4-756607911180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:15.130606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba341138-f41c-41d5-82b4-756607911180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:01:15.132951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba341138-f41c-41d5-82b4-756607911180 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:01:15.146039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1102/2210] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:01:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:01:15.529645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57b242-0566-4852-a272-723928c55c18 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:01:15.533387 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-8e57b242-0566-4852-a272-723928c55c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:15.663610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57b242-0566-4852-a272-723928c55c18 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:01:15.676490 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a874756-33a2-46aa-96f7-a0141c7d2ccd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ae9a6f1ac304e21b8d3115f2a0baf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:01:15.676844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a874756-33a2-46aa-96f7-a0141c7d2ccd service placement] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:15.677165 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a874756-33a2-46aa-96f7-a0141c7d2ccd service placement] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:15.677522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a874756-33a2-46aa-96f7-a0141c7d2ccd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:15.682060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0a874756-33a2-46aa-96f7-a0141c7d2ccd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:15.705150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a874756-33a2-46aa-96f7-a0141c7d2ccd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:01:15.707748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a874756-33a2-46aa-96f7-a0141c7d2ccd service placement] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:01:15.720238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1109/2211] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:01:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:01:16.632760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba341138-f41c-41d5-82b4-756607911180 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:01:16.638894 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ba341138-f41c-41d5-82b4-756607911180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:16.754531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba341138-f41c-41d5-82b4-756607911180 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:01:16.762847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c790f-d4fa-4671-867f-e155ca35bbd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:01:16.763519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c790f-d4fa-4671-867f-e155ca35bbd2 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:16.763802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c790f-d4fa-4671-867f-e155ca35bbd2 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:16.764076 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c790f-d4fa-4671-867f-e155ca35bbd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:16.767269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4e8c790f-d4fa-4671-867f-e155ca35bbd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:16.790165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c790f-d4fa-4671-867f-e155ca35bbd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:01:16.793330 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c790f-d4fa-4671-867f-e155ca35bbd2 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:01:16.810207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1103/2212] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:01:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:01:16.999675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a874756-33a2-46aa-96f7-a0141c7d2ccd service placement] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:01:17.010127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a874756-33a2-46aa-96f7-a0141c7d2ccd service placement] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:01:17.017509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecf585-9d33-4eee-8cac-cc937e57a041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:01:17.018843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecf585-9d33-4eee-8cac-cc937e57a041 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:17.019328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecf585-9d33-4eee-8cac-cc937e57a041 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:17.019668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecf585-9d33-4eee-8cac-cc937e57a041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:17.025923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7aecf585-9d33-4eee-8cac-cc937e57a041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:17.034999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecf585-9d33-4eee-8cac-cc937e57a041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:01:17.038044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecf585-9d33-4eee-8cac-cc937e57a041 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:01:17.061997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1110/2213] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:01:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:01:19.048463 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c790f-d4fa-4671-867f-e155ca35bbd2 service nova] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:01:19.062728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c790f-d4fa-4671-867f-e155ca35bbd2 service nova] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:01:19.069491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f5eda-c904-454d-8dce-27c50df38781 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:01:19.070336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f5eda-c904-454d-8dce-27c50df38781 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:19.070652 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f5eda-c904-454d-8dce-27c50df38781 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:19.071098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f5eda-c904-454d-8dce-27c50df38781 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:19.073186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c93f5eda-c904-454d-8dce-27c50df38781 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:19.090278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f5eda-c904-454d-8dce-27c50df38781 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:01:19.091895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f5eda-c904-454d-8dce-27c50df38781 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:01:19.107468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1104/2214] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:01:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:01:20.627464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecf585-9d33-4eee-8cac-cc937e57a041 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:01:20.640263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecf585-9d33-4eee-8cac-cc937e57a041 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:01:20.646309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c0f6c-4f8f-4aff-bee7-e8e4498c8f8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:01:20.646981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c0f6c-4f8f-4aff-bee7-e8e4498c8f8b service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:20.647265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c0f6c-4f8f-4aff-bee7-e8e4498c8f8b service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:20.647540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c0f6c-4f8f-4aff-bee7-e8e4498c8f8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:20.649323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-769c0f6c-4f8f-4aff-bee7-e8e4498c8f8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:20.707106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c0f6c-4f8f-4aff-bee7-e8e4498c8f8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:01:20.708695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c0f6c-4f8f-4aff-bee7-e8e4498c8f8b service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:01:20.717670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1111/2215] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:01:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:01:31.747802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f5eda-c904-454d-8dce-27c50df38781 service nova] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:01:31.749197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c93f5eda-c904-454d-8dce-27c50df38781 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:31.804440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f5eda-c904-454d-8dce-27c50df38781 service nova] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:01:31.808690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a6433-4679-43ea-95c4-329014c37322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:01:31.809331 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a6433-4679-43ea-95c4-329014c37322 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:31.809609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a6433-4679-43ea-95c4-329014c37322 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:31.809881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a6433-4679-43ea-95c4-329014c37322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:31.811667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-251a6433-4679-43ea-95c4-329014c37322 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:31.870792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a6433-4679-43ea-95c4-329014c37322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:01:31.872270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a6433-4679-43ea-95c4-329014c37322 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:01:31.881153 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1105/2216] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:01:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:01:36.643967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c0f6c-4f8f-4aff-bee7-e8e4498c8f8b service nova] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:01:36.646164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-769c0f6c-4f8f-4aff-bee7-e8e4498c8f8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:36.672133 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c0f6c-4f8f-4aff-bee7-e8e4498c8f8b service nova] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:01:36.685074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f85ed6-b24e-44d3-93ea-b6cf69b18c54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:01:36.685780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85ed6-b24e-44d3-93ea-b6cf69b18c54 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:36.686142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85ed6-b24e-44d3-93ea-b6cf69b18c54 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:36.686412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85ed6-b24e-44d3-93ea-b6cf69b18c54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:36.688523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-19f85ed6-b24e-44d3-93ea-b6cf69b18c54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:36.752913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f85ed6-b24e-44d3-93ea-b6cf69b18c54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:01:36.754736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f85ed6-b24e-44d3-93ea-b6cf69b18c54 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:01:36.764919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1112/2217] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:01:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:01:38.819960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2d4c4-6842-4fd7-b3e8-e51ef5117ac6 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:38.820594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2d4c4-6842-4fd7-b3e8-e51ef5117ac6 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:38.820594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2d4c4-6842-4fd7-b3e8-e51ef5117ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:38.839522 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-45f2d4c4-6842-4fd7-b3e8-e51ef5117ac6 None None] MFA Rules not processed for user `20fd230cc30f4980bedebc75c620d707`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:01:38.840763 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-45f2d4c4-6842-4fd7-b3e8-e51ef5117ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:38.850908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1106/2218] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:01:40.381503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f7e3a-ad74-4039-9b4f-da119b8adc27 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:40.382093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f7e3a-ad74-4039-9b4f-da119b8adc27 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:40.382093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f7e3a-ad74-4039-9b4f-da119b8adc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:40.401031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-cf2f7e3a-ad74-4039-9b4f-da119b8adc27 None None] MFA Rules not processed for user `a37f5d5ddd9845fbaf7d5645c31b2d40`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:01:40.402274 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-cf2f7e3a-ad74-4039-9b4f-da119b8adc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:40.411555 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1113/2219] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:01:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:01:50.797562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2d4c4-6842-4fd7-b3e8-e51ef5117ac6 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:01:50.820617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2d4c4-6842-4fd7-b3e8-e51ef5117ac6 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:01:50.831021 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab118b-2041-411c-bda2-d993d730d06a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:01:50.832410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab118b-2041-411c-bda2-d993d730d06a service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:50.834139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab118b-2041-411c-bda2-d993d730d06a service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:50.834139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab118b-2041-411c-bda2-d993d730d06a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:50.837476 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-baab118b-2041-411c-bda2-d993d730d06a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:50.853715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab118b-2041-411c-bda2-d993d730d06a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:01:50.855774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab118b-2041-411c-bda2-d993d730d06a service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:01:50.868173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1107/2220] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:01:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:01:52.318131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f7e3a-ad74-4039-9b4f-da119b8adc27 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:01:52.337404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f7e3a-ad74-4039-9b4f-da119b8adc27 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:01:52.351549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66c047-d534-4c7a-a286-7471cf369786 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:01:52.353518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c047-d534-4c7a-a286-7471cf369786 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:52.353962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c047-d534-4c7a-a286-7471cf369786 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:52.354393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c047-d534-4c7a-a286-7471cf369786 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:52.356554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6c66c047-d534-4c7a-a286-7471cf369786 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:52.375204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66c047-d534-4c7a-a286-7471cf369786 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:01:52.377451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66c047-d534-4c7a-a286-7471cf369786 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:01:52.400331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1114/2221] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:01:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:01:56.272342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e86ba-7947-4485-b4cc-1ca9d8d756a9 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:01:56.272941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e86ba-7947-4485-b4cc-1ca9d8d756a9 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:01:56.272941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e86ba-7947-4485-b4cc-1ca9d8d756a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:01:56.305458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-ea8e86ba-7947-4485-b4cc-1ca9d8d756a9 None None] MFA Rules not processed for user `648ae92126944b50a22ad780e92df914`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:01:56.307804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ea8e86ba-7947-4485-b4cc-1ca9d8d756a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:01:56.320519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1108/2222] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:01:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:02:40.478035 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aced76-1bbf-4acc-83f3-74f238417a01 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:40.479003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aced76-1bbf-4acc-83f3-74f238417a01 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:40.480220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aced76-1bbf-4acc-83f3-74f238417a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:40.506197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-99aced76-1bbf-4acc-83f3-74f238417a01 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:02:40.507388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-99aced76-1bbf-4acc-83f3-74f238417a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:40.579384 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1115/2223] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:02:40.695408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e86ba-7947-4485-b4cc-1ca9d8d756a9 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:40.697769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ea8e86ba-7947-4485-b4cc-1ca9d8d756a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:40.822215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e86ba-7947-4485-b4cc-1ca9d8d756a9 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:40.829049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cb64f-514e-40a0-99c5-2b82b4d44147 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:40.829790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cb64f-514e-40a0-99c5-2b82b4d44147 service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:40.830142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cb64f-514e-40a0-99c5-2b82b4d44147 service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:40.830479 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cb64f-514e-40a0-99c5-2b82b4d44147 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:40.840953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cb64f-514e-40a0-99c5-2b82b4d44147 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:40.842634 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cb64f-514e-40a0-99c5-2b82b4d44147 service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:40.852583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1109/2224] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:02:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 14:02:44.085557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e11bd-96b9-4fb5-a317-78d8833becba None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:44.086155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e11bd-96b9-4fb5-a317-78d8833becba None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:44.087108 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e11bd-96b9-4fb5-a317-78d8833becba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:44.129949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-fc7e11bd-96b9-4fb5-a317-78d8833becba None None] MFA Rules not processed for user `a27e83f9f5fb4a0b99e7fac6ab90b3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:02:44.132432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-fc7e11bd-96b9-4fb5-a317-78d8833becba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:44.214411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1116/2225] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:02:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:02:52.404874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cb64f-514e-40a0-99c5-2b82b4d44147 service swift] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:52.420464 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cb64f-514e-40a0-99c5-2b82b4d44147 service swift] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:52.447052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c767702-a029-4246-b42e-ed3f7ad9bc50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:52.447052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767702-a029-4246-b42e-ed3f7ad9bc50 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:52.447052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767702-a029-4246-b42e-ed3f7ad9bc50 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:52.447052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767702-a029-4246-b42e-ed3f7ad9bc50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:52.453979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0c767702-a029-4246-b42e-ed3f7ad9bc50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:52.594143 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c767702-a029-4246-b42e-ed3f7ad9bc50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:52.595680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c767702-a029-4246-b42e-ed3f7ad9bc50 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:52.607158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1110/2226] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:02:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:02:58.283320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e11bd-96b9-4fb5-a317-78d8833becba None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:58.284943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-fc7e11bd-96b9-4fb5-a317-78d8833becba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:58.308697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e11bd-96b9-4fb5-a317-78d8833becba None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:58.321995 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:58.323418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:58.323854 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:58.324425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] PATH_INFO: `/v3/users/20fd230cc30f4980bedebc75c620d707` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:58.325917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=20fd230cc30f4980bedebc75c620d707)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:58.327939 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:58.394138 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20fd230cc30f4980bedebc75c620d707'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:02:58.415247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20fd230cc30f4980bedebc75c620d707'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:02:58.425597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20fd230cc30f4980bedebc75c620d707'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:02:58.426968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20fd230cc30f4980bedebc75c620d707'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:02:58.440392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-24ae8baf-9e46-4211-9a34-d3c894646b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20fd230cc30f4980bedebc75c620d707'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:02:58.447375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1117/2227] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:02:58 2022] DELETE /identity/v3/users/20fd230cc30f4980bedebc75c620d707 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:02:58.654106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c767702-a029-4246-b42e-ed3f7ad9bc50 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:58.673753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c767702-a029-4246-b42e-ed3f7ad9bc50 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:58.761765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:58.762973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:58.763256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:58.763519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] PATH_INFO: `/v3/projects/ef89791b177f4bf0ba916f133c79f33a` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:58.766238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef89791b177f4bf0ba916f133c79f33a)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:58.769500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:58.851713 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef89791b177f4bf0ba916f133c79f33a', 'request_id': 'req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:02:58.860180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef89791b177f4bf0ba916f133c79f33a', 'request_id': 'req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:02:58.873008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef89791b177f4bf0ba916f133c79f33a', 'request_id': 'req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:02:58.894790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] The token cache is being invalidate because project ef89791b177f4bf0ba916f133c79f33a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:02:58.895091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:02:58.895743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9b79d1cb-bbfe-4cd8-980d-4e53faca0070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:02:58.897311 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1111/2228] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:02:58 2022] DELETE /identity/v3/projects/ef89791b177f4bf0ba916f133c79f33a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:02:58.905289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1e8a7-c3ad-4b0c-97a3-5e6c249a4d4a None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:58.906089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1e8a7-c3ad-4b0c-97a3-5e6c249a4d4a None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:58.906319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1e8a7-c3ad-4b0c-97a3-5e6c249a4d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:58.929905 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2da1e8a7-c3ad-4b0c-97a3-5e6c249a4d4a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:02:58.929905 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2da1e8a7-c3ad-4b0c-97a3-5e6c249a4d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:58.992665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8cf76-8287-4c0e-b837-1507a83c4499 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:58.993085 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8cf76-8287-4c0e-b837-1507a83c4499 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:58.993256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8cf76-8287-4c0e-b837-1507a83c4499 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:58.995116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1112/2229] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 14:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 14:02:59.019737 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a368e98-4f94-4d94-a29e-243bf30cafab None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.020328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a368e98-4f94-4d94-a29e-243bf30cafab None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.020757 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a368e98-4f94-4d94-a29e-243bf30cafab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.034102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1118/2230] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:02:59.043284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f9261-4bb4-459e-915e-9fa2b6536c4d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.043708 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7a368e98-4f94-4d94-a29e-243bf30cafab None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:02:59.044980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f9261-4bb4-459e-915e-9fa2b6536c4d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.045299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f9261-4bb4-459e-915e-9fa2b6536c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.046395 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7a368e98-4f94-4d94-a29e-243bf30cafab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:59.067200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0e4f9261-4bb4-459e-915e-9fa2b6536c4d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:02:59.068468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0e4f9261-4bb4-459e-915e-9fa2b6536c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:59.076890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1119/2231] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:02:59.083772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f9261-4bb4-459e-915e-9fa2b6536c4d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.094282 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f9261-4bb4-459e-915e-9fa2b6536c4d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.101420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c65673-ba98-459e-89a1-d47a84874fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.102097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c65673-ba98-459e-89a1-d47a84874fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.102425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c65673-ba98-459e-89a1-d47a84874fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.102624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c65673-ba98-459e-89a1-d47a84874fd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.103982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c65673-ba98-459e-89a1-d47a84874fd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:02:59.103982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c65673-ba98-459e-89a1-d47a84874fd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.105413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c65673-ba98-459e-89a1-d47a84874fd5 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.121341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1120/2232] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 14:02:59.127193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c65673-ba98-459e-89a1-d47a84874fd5 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.138632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c65673-ba98-459e-89a1-d47a84874fd5 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.141278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1113/2233] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:02:59.148312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a6623-14e4-4153-81e6-a45566934597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.148586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a6623-14e4-4153-81e6-a45566934597 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.149143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a6623-14e4-4153-81e6-a45566934597 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.149418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a6623-14e4-4153-81e6-a45566934597 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.150111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a6623-14e4-4153-81e6-a45566934597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.152315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a6623-14e4-4153-81e6-a45566934597 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.154412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a368e98-4f94-4d94-a29e-243bf30cafab None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.156426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7a368e98-4f94-4d94-a29e-243bf30cafab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:59.196280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1121/2234] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:02:59 2022] POST /identity/v3/projects => generated 403 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:02:59.200785 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a6623-14e4-4153-81e6-a45566934597 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.220938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a368e98-4f94-4d94-a29e-243bf30cafab None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.271275 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a6623-14e4-4153-81e6-a45566934597 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.273087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc69ad-2814-4a67-b402-bbf43900c1c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4f45e859d84ceaa5a01477e83e098e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.273745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc69ad-2814-4a67-b402-bbf43900c1c1 service glance] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.274028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc69ad-2814-4a67-b402-bbf43900c1c1 service glance] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.274301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc69ad-2814-4a67-b402-bbf43900c1c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.276547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa313e99-74cc-4a20-9206-988168c6164e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.277216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa313e99-74cc-4a20-9206-988168c6164e admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.277503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa313e99-74cc-4a20-9206-988168c6164e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.277877 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa313e99-74cc-4a20-9206-988168c6164e admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.278533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa313e99-74cc-4a20-9206-988168c6164e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.281228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa313e99-74cc-4a20-9206-988168c6164e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.305321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1122/2235] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:02:59 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:02:59.309940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa313e99-74cc-4a20-9206-988168c6164e admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.319735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa313e99-74cc-4a20-9206-988168c6164e admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.324493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab19b4-f7e7-48ab-8e06-2abe474deb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.325093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab19b4-f7e7-48ab-8e06-2abe474deb30 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.325452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab19b4-f7e7-48ab-8e06-2abe474deb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.325729 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab19b4-f7e7-48ab-8e06-2abe474deb30 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.326743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab19b4-f7e7-48ab-8e06-2abe474deb30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.328730 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab19b4-f7e7-48ab-8e06-2abe474deb30 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.339642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1123/2236] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:02:59.344558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab19b4-f7e7-48ab-8e06-2abe474deb30 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.353626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc69ad-2814-4a67-b402-bbf43900c1c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.355362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc69ad-2814-4a67-b402-bbf43900c1c1 service glance] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.355881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab19b4-f7e7-48ab-8e06-2abe474deb30 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.361877 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1592d-0faa-49f2-b831-c7d255a7ddb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.362807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1592d-0faa-49f2-b831-c7d255a7ddb4 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.362978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1592d-0faa-49f2-b831-c7d255a7ddb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.363235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1592d-0faa-49f2-b831-c7d255a7ddb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.363972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1592d-0faa-49f2-b831-c7d255a7ddb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.365564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1592d-0faa-49f2-b831-c7d255a7ddb4 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.368739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1114/2237] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:02:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:02:59.382942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1124/2238] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:02:59.387310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc69ad-2814-4a67-b402-bbf43900c1c1 service glance] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.398267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc69ad-2814-4a67-b402-bbf43900c1c1 service glance] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.403083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4842b9-ffa9-4116-9a5c-c15107085210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.403982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4842b9-ffa9-4116-9a5c-c15107085210 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.404269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4842b9-ffa9-4116-9a5c-c15107085210 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.404532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4842b9-ffa9-4116-9a5c-c15107085210 admin admin] PATH_INFO: `/v3/projects/134bce2232294349a30cfd477ee43524/users/051bd4d79ecd43ba97d9ca6203f23e2b/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.422491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4842b9-ffa9-4116-9a5c-c15107085210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=134bce2232294349a30cfd477ee43524, user_id=051bd4d79ecd43ba97d9ca6203f23e2b, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.425278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4842b9-ffa9-4116-9a5c-c15107085210 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.449939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1115/2239] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:02:59 2022] PUT /identity/v3/projects/134bce2232294349a30cfd477ee43524/users/051bd4d79ecd43ba97d9ca6203f23e2b/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:02:59.450195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d13d15-a8cb-4382-90b3-fba1afc3af4d None None] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.450195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d13d15-a8cb-4382-90b3-fba1afc3af4d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.450195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d13d15-a8cb-4382-90b3-fba1afc3af4d None None] PATH_INFO: `/` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.450195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1125/2240] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 14:02:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 14:02:59.454929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197af31a-baac-4c2e-a25f-50932e6d497c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.455299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197af31a-baac-4c2e-a25f-50932e6d497c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.455576 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197af31a-baac-4c2e-a25f-50932e6d497c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.455713 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8de4c-3ada-4544-a9e0-cf77f406c513 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.455890 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8de4c-3ada-4544-a9e0-cf77f406c513 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.456182 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8de4c-3ada-4544-a9e0-cf77f406c513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.485493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1ff8de4c-3ada-4544-a9e0-cf77f406c513 None None] MFA Rules not processed for user `8f784cc461ab48ad92140680758a3704`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:02:59.486957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1ff8de4c-3ada-4544-a9e0-cf77f406c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:59.487229 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-197af31a-baac-4c2e-a25f-50932e6d497c None None] MFA Rules not processed for user `051bd4d79ecd43ba97d9ca6203f23e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:02:59.488769 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-197af31a-baac-4c2e-a25f-50932e6d497c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:59.569341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1116/2241] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:02:59.587575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8de4c-3ada-4544-a9e0-cf77f406c513 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.590202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1ff8de4c-3ada-4544-a9e0-cf77f406c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:59.594475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1126/2242] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:02:59.600830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197af31a-baac-4c2e-a25f-50932e6d497c None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.606774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8de4c-3ada-4544-a9e0-cf77f406c513 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.611025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd574fd-baa5-474e-8511-98403de81c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.611970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd574fd-baa5-474e-8511-98403de81c8b service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.612328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd574fd-baa5-474e-8511-98403de81c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.612668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd574fd-baa5-474e-8511-98403de81c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.625905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd574fd-baa5-474e-8511-98403de81c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.627992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd574fd-baa5-474e-8511-98403de81c8b service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.654315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1117/2243] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:02:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:02:59.721095 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197af31a-baac-4c2e-a25f-50932e6d497c None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.729608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af4676-6546-4561-8058-e79a427328bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.730813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af4676-6546-4561-8058-e79a427328bb admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.731322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af4676-6546-4561-8058-e79a427328bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.731821 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af4676-6546-4561-8058-e79a427328bb admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.733155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af4676-6546-4561-8058-e79a427328bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.737360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af4676-6546-4561-8058-e79a427328bb admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.754420 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc598f-b6fc-4d9a-8bf2-24d084a7d6e5 None None] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.755107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc598f-b6fc-4d9a-8bf2-24d084a7d6e5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.755583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc598f-b6fc-4d9a-8bf2-24d084a7d6e5 None None] PATH_INFO: `/` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.757793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1118/2244] 10.222.0.79 () {54 vars in 921 bytes} [Fri May 13 14:02:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 14:02:59.765096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5854697-6d1f-440e-9f86-a7c6a84c25a5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.765671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5854697-6d1f-440e-9f86-a7c6a84c25a5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.766243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5854697-6d1f-440e-9f86-a7c6a84c25a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.786971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1127/2245] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:02:59 2022] POST /identity/v3/projects => generated 403 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:02:59.787544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c5854697-6d1f-440e-9f86-a7c6a84c25a5 None None] MFA Rules not processed for user `24ad0dd9d265425087be2be8d245b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:02:59.788759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c5854697-6d1f-440e-9f86-a7c6a84c25a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:02:59.791315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af4676-6546-4561-8058-e79a427328bb admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.858588 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af4676-6546-4561-8058-e79a427328bb admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.860554 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1119/2246] 10.222.0.79 () {58 vars in 1018 bytes} [Fri May 13 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:02:59.863917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f899a-a18a-46b4-a1bd-8f086dadc8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.864601 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f899a-a18a-46b4-a1bd-8f086dadc8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.865037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f899a-a18a-46b4-a1bd-8f086dadc8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.865319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f899a-a18a-46b4-a1bd-8f086dadc8b4 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.866023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f899a-a18a-46b4-a1bd-8f086dadc8b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.867999 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f899a-a18a-46b4-a1bd-8f086dadc8b4 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.872009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5854697-6d1f-440e-9f86-a7c6a84c25a5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.891312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1128/2247] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:02:59 2022] POST /identity/v3/users => generated 308 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:02:59.895676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f899a-a18a-46b4-a1bd-8f086dadc8b4 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.906244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f899a-a18a-46b4-a1bd-8f086dadc8b4 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.911346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fa26a-2536-4d42-b2e6-3d17a9c47b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.912100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fa26a-2536-4d42-b2e6-3d17a9c47b06 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.912338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fa26a-2536-4d42-b2e6-3d17a9c47b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.912622 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fa26a-2536-4d42-b2e6-3d17a9c47b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.913352 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fa26a-2536-4d42-b2e6-3d17a9c47b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.915012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fa26a-2536-4d42-b2e6-3d17a9c47b06 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.930188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1129/2248] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:02:59.931625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fa26a-2536-4d42-b2e6-3d17a9c47b06 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:02:59.932992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5854697-6d1f-440e-9f86-a7c6a84c25a5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.938162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3edc0-85da-4f70-b9e3-e493aa5cc5f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.939443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3edc0-85da-4f70-b9e3-e493aa5cc5f5 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.939693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3edc0-85da-4f70-b9e3-e493aa5cc5f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.940018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3edc0-85da-4f70-b9e3-e493aa5cc5f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.944408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fa26a-2536-4d42-b2e6-3d17a9c47b06 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:02:59.949272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510445a6-ac3a-4bda-bc21-cf591f1684ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:02:59.950170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510445a6-ac3a-4bda-bc21-cf591f1684ad admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:02:59.950451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510445a6-ac3a-4bda-bc21-cf591f1684ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:02:59.950747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510445a6-ac3a-4bda-bc21-cf591f1684ad admin admin] PATH_INFO: `/v3/projects/9d5bd377a2534354a68fbabf10e8c481/users/75f844916f2645dfb3bcf118d6b09567/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:02:59.954361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3edc0-85da-4f70-b9e3-e493aa5cc5f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.956063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3edc0-85da-4f70-b9e3-e493aa5cc5f5 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.967105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510445a6-ac3a-4bda-bc21-cf591f1684ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5bd377a2534354a68fbabf10e8c481, user_id=75f844916f2645dfb3bcf118d6b09567, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:02:59.968053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1120/2249] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:02:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:02:59.969911 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510445a6-ac3a-4bda-bc21-cf591f1684ad admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:02:59.985513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1130/2250] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:02:59 2022] PUT /identity/v3/projects/9d5bd377a2534354a68fbabf10e8c481/users/75f844916f2645dfb3bcf118d6b09567/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:02:59.989705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3edc0-85da-4f70-b9e3-e493aa5cc5f5 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:00.000792 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3edc0-85da-4f70-b9e3-e493aa5cc5f5 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:00.064354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e165b93-1819-4fe3-83c0-ca8aa080070b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:00.065013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e165b93-1819-4fe3-83c0-ca8aa080070b admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:00.065296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e165b93-1819-4fe3-83c0-ca8aa080070b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:00.065560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e165b93-1819-4fe3-83c0-ca8aa080070b admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:00.066579 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e165b93-1819-4fe3-83c0-ca8aa080070b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:00.067952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e165b93-1819-4fe3-83c0-ca8aa080070b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:00.079030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1121/2251] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:00.083295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510445a6-ac3a-4bda-bc21-cf591f1684ad admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:00.094866 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510445a6-ac3a-4bda-bc21-cf591f1684ad admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:00.099911 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d06b3-f6d8-4aa9-845a-ae2dfa16f2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:00.100812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d06b3-f6d8-4aa9-845a-ae2dfa16f2a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:00.101094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d06b3-f6d8-4aa9-845a-ae2dfa16f2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:00.101379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d06b3-f6d8-4aa9-845a-ae2dfa16f2a9 admin admin] PATH_INFO: `/v3/projects/9d5bd377a2534354a68fbabf10e8c481/users/75f844916f2645dfb3bcf118d6b09567/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:00.103713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d06b3-f6d8-4aa9-845a-ae2dfa16f2a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5bd377a2534354a68fbabf10e8c481, user_id=75f844916f2645dfb3bcf118d6b09567, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:00.106321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d06b3-f6d8-4aa9-845a-ae2dfa16f2a9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:00.118139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1131/2252] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:00 2022] PUT /identity/v3/projects/9d5bd377a2534354a68fbabf10e8c481/users/75f844916f2645dfb3bcf118d6b09567/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:00.135669 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45174f-ae90-42e0-84a7-f254b5af9a2f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:00.135862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45174f-ae90-42e0-84a7-f254b5af9a2f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:00.136144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45174f-ae90-42e0-84a7-f254b5af9a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:00.184746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7f45174f-ae90-42e0-84a7-f254b5af9a2f None None] MFA Rules not processed for user `75f844916f2645dfb3bcf118d6b09567`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:00.185916 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7f45174f-ae90-42e0-84a7-f254b5af9a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:00.295752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1122/2253] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:00 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:00.307386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d06b3-f6d8-4aa9-845a-ae2dfa16f2a9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:00.309833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ad4d06b3-f6d8-4aa9-845a-ae2dfa16f2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:00.380392 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d06b3-f6d8-4aa9-845a-ae2dfa16f2a9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:00.385794 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904ebc6-2c3b-4cc2-935b-66d25e6c4e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:00.386590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904ebc6-2c3b-4cc2-935b-66d25e6c4e86 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:00.387320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904ebc6-2c3b-4cc2-935b-66d25e6c4e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:00.387320 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904ebc6-2c3b-4cc2-935b-66d25e6c4e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:00.410805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904ebc6-2c3b-4cc2-935b-66d25e6c4e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:00.412629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904ebc6-2c3b-4cc2-935b-66d25e6c4e86 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:00.423224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1132/2254] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:00 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:00.593531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45174f-ae90-42e0-84a7-f254b5af9a2f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:00.666569 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45174f-ae90-42e0-84a7-f254b5af9a2f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:00.671973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:00.672841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:00.673123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:00.673401 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] PATH_INFO: `/v3/users/051bd4d79ecd43ba97d9ca6203f23e2b` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:00.674605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=051bd4d79ecd43ba97d9ca6203f23e2b)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:00.676619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:00.739440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051bd4d79ecd43ba97d9ca6203f23e2b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:00.740873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051bd4d79ecd43ba97d9ca6203f23e2b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:00.766588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051bd4d79ecd43ba97d9ca6203f23e2b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:00.767394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051bd4d79ecd43ba97d9ca6203f23e2b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:00.775167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051bd4d79ecd43ba97d9ca6203f23e2b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:00.806879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1123/2255] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:00 2022] DELETE /identity/v3/users/051bd4d79ecd43ba97d9ca6203f23e2b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:00.813522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0e084-1aa2-4bde-a9b3-9074ccde95bb None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:00.813988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0e084-1aa2-4bde-a9b3-9074ccde95bb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:00.814452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0e084-1aa2-4bde-a9b3-9074ccde95bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:00.851777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1eb0e084-1aa2-4bde-a9b3-9074ccde95bb None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:00.854947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1eb0e084-1aa2-4bde-a9b3-9074ccde95bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:00.948041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1133/2256] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:00.960474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:00.962403 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:01.092776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e2295-4864-45b7-a2fc-c97554104abd admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:01.103734 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364938f2-cccf-4a39-809b-f64332986ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:01.104957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364938f2-cccf-4a39-809b-f64332986ffc service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:01.105705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364938f2-cccf-4a39-809b-f64332986ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:01.106009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364938f2-cccf-4a39-809b-f64332986ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:01.124933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364938f2-cccf-4a39-809b-f64332986ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:01.129043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364938f2-cccf-4a39-809b-f64332986ffc service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:01.147196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1124/2257] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:01.704031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0e084-1aa2-4bde-a9b3-9074ccde95bb None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:01.707793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1eb0e084-1aa2-4bde-a9b3-9074ccde95bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:01.731244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0e084-1aa2-4bde-a9b3-9074ccde95bb None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:01.753664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:01.755625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:01.756324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:01.756919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] PATH_INFO: `/v3/projects/134bce2232294349a30cfd477ee43524` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:01.759983 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=134bce2232294349a30cfd477ee43524)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:01.765929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:01.914242 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '134bce2232294349a30cfd477ee43524', 'request_id': 'req-7d70b137-3d10-48a2-93c8-8c16436935bc'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:01.927702 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '134bce2232294349a30cfd477ee43524', 'request_id': 'req-7d70b137-3d10-48a2-93c8-8c16436935bc'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:01.953895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '134bce2232294349a30cfd477ee43524', 'request_id': 'req-7d70b137-3d10-48a2-93c8-8c16436935bc'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:01.977484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] The token cache is being invalidate because project 134bce2232294349a30cfd477ee43524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:01.978574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:01.980404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:01.984960 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1134/2258] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:01 2022] DELETE /identity/v3/projects/134bce2232294349a30cfd477ee43524 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:01.991012 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364938f2-cccf-4a39-809b-f64332986ffc service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:02.023515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364938f2-cccf-4a39-809b-f64332986ffc service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:02.121391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:02.122855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:02.124777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:02.125273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] PATH_INFO: `/v3/users/75f844916f2645dfb3bcf118d6b09567` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:02.127784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75f844916f2645dfb3bcf118d6b09567)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:02.130610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:02.246402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f844916f2645dfb3bcf118d6b09567'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:02.248105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f844916f2645dfb3bcf118d6b09567'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:02.285178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f844916f2645dfb3bcf118d6b09567'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:02.287100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f844916f2645dfb3bcf118d6b09567'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:02.303005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e6808239-deff-4e3e-8f2d-b46249493df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f844916f2645dfb3bcf118d6b09567'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:02.330857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1125/2259] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:01 2022] DELETE /identity/v3/users/75f844916f2645dfb3bcf118d6b09567 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:02.797770 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:02.800103 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:02.819923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70b137-3d10-48a2-93c8-8c16436935bc admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:02.920375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:02.921857 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:02.922132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:02.922592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] PATH_INFO: `/v3/projects/9d5bd377a2534354a68fbabf10e8c481` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:02.924264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d5bd377a2534354a68fbabf10e8c481)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:02.926237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:02.947378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb66042-2dba-4ade-8915-2740c573da55 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:02.947744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb66042-2dba-4ade-8915-2740c573da55 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:02.948237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb66042-2dba-4ade-8915-2740c573da55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:02.977068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-eeb66042-2dba-4ade-8915-2740c573da55 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:02.979146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-eeb66042-2dba-4ade-8915-2740c573da55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:03.010837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d5bd377a2534354a68fbabf10e8c481', 'request_id': 'req-a68d03db-3512-46c5-8de2-b31e256aeb91'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:03.021613 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d5bd377a2534354a68fbabf10e8c481', 'request_id': 'req-a68d03db-3512-46c5-8de2-b31e256aeb91'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:03.035385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d5bd377a2534354a68fbabf10e8c481', 'request_id': 'req-a68d03db-3512-46c5-8de2-b31e256aeb91'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:03.053198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] The token cache is being invalidate because project 9d5bd377a2534354a68fbabf10e8c481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:03.053755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:03.055015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a68d03db-3512-46c5-8de2-b31e256aeb91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:03.057780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1135/2260] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:02 2022] DELETE /identity/v3/projects/9d5bd377a2534354a68fbabf10e8c481 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:03.058691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1126/2261] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 14:03:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:03.064089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9794f9-465c-4b44-9945-9a383d2e8b93 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:03.064761 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb66042-2dba-4ade-8915-2740c573da55 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:03.065124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9794f9-465c-4b44-9945-9a383d2e8b93 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:03.065639 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9794f9-465c-4b44-9945-9a383d2e8b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:03.066284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-eeb66042-2dba-4ade-8915-2740c573da55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:03.090153 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-7f9794f9-465c-4b44-9945-9a383d2e8b93 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:03.091610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7f9794f9-465c-4b44-9945-9a383d2e8b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:03.190128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb66042-2dba-4ade-8915-2740c573da55 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:03.210301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1136/2262] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:03.226329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f362-ca32-4872-a916-815205f81c64 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:03.227167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f362-ca32-4872-a916-815205f81c64 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:03.232057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f362-ca32-4872-a916-815205f81c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:03.265580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0e47f362-ca32-4872-a916-815205f81c64 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:03.267984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7206b59-4bc6-47f7-91dd-dcd658f5c5b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:03.268265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0e47f362-ca32-4872-a916-815205f81c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:03.268740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7206b59-4bc6-47f7-91dd-dcd658f5c5b1 service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:03.269018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7206b59-4bc6-47f7-91dd-dcd658f5c5b1 service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:03.269426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7206b59-4bc6-47f7-91dd-dcd658f5c5b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:03.271945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f7206b59-4bc6-47f7-91dd-dcd658f5c5b1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:03.288317 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1137/2263] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:03.293459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47f362-ca32-4872-a916-815205f81c64 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:03.307378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47f362-ca32-4872-a916-815205f81c64 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:03.314052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67390550-0c8b-46b9-970c-23e84cf68c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:03.315174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67390550-0c8b-46b9-970c-23e84cf68c35 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:03.316156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67390550-0c8b-46b9-970c-23e84cf68c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:03.316156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67390550-0c8b-46b9-970c-23e84cf68c35 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:03.317326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67390550-0c8b-46b9-970c-23e84cf68c35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:03.317768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67390550-0c8b-46b9-970c-23e84cf68c35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:03.320193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67390550-0c8b-46b9-970c-23e84cf68c35 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:03.344093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1138/2264] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:03:03.345849 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67390550-0c8b-46b9-970c-23e84cf68c35 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:03.359752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67390550-0c8b-46b9-970c-23e84cf68c35 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:03.366251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f022c2-8c87-444c-8011-9595d3d5aa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:03.367183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f022c2-8c87-444c-8011-9595d3d5aa98 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:03.367462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f022c2-8c87-444c-8011-9595d3d5aa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:03.367708 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7206b59-4bc6-47f7-91dd-dcd658f5c5b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:03.367912 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f022c2-8c87-444c-8011-9595d3d5aa98 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:03.368673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f022c2-8c87-444c-8011-9595d3d5aa98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:03.369469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7206b59-4bc6-47f7-91dd-dcd658f5c5b1 service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:03.371408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f022c2-8c87-444c-8011-9595d3d5aa98 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:03.381434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1127/2265] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:03:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 14:03:03.429255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1139/2266] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:03 2022] POST /identity/v3/projects => generated 379 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:03.438572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7206b59-4bc6-47f7-91dd-dcd658f5c5b1 service swift] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:03.451663 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7206b59-4bc6-47f7-91dd-dcd658f5c5b1 service swift] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:03.594049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09f7b2-c15a-42a4-899f-8ac47e536a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:03.594962 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09f7b2-c15a-42a4-899f-8ac47e536a11 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:03.595347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09f7b2-c15a-42a4-899f-8ac47e536a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:03.595730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09f7b2-c15a-42a4-899f-8ac47e536a11 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:03.597194 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09f7b2-c15a-42a4-899f-8ac47e536a11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:03.599917 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09f7b2-c15a-42a4-899f-8ac47e536a11 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:03.642163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1128/2267] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:03 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:03.642428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f022c2-8c87-444c-8011-9595d3d5aa98 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:03.683243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f022c2-8c87-444c-8011-9595d3d5aa98 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:03.722352 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399d884-6ee3-4247-a436-1221a372e7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:03.724042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399d884-6ee3-4247-a436-1221a372e7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:03.724536 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399d884-6ee3-4247-a436-1221a372e7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:03.724985 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399d884-6ee3-4247-a436-1221a372e7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:03.726359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399d884-6ee3-4247-a436-1221a372e7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:03.729348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399d884-6ee3-4247-a436-1221a372e7f7 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:03.761508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1140/2268] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:03.770781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09f7b2-c15a-42a4-899f-8ac47e536a11 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:03.797561 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09f7b2-c15a-42a4-899f-8ac47e536a11 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:03.814104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc20c85-e6e5-4fdd-9d73-b5130d4ab4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:03.815933 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc20c85-e6e5-4fdd-9d73-b5130d4ab4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:03.816554 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc20c85-e6e5-4fdd-9d73-b5130d4ab4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:03.816956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc20c85-e6e5-4fdd-9d73-b5130d4ab4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:03.818102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc20c85-e6e5-4fdd-9d73-b5130d4ab4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:03.823338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc20c85-e6e5-4fdd-9d73-b5130d4ab4b3 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:03.859315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399d884-6ee3-4247-a436-1221a372e7f7 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:03.860135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1129/2269] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:03.883150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399d884-6ee3-4247-a436-1221a372e7f7 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:03.889500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07dcf9-5e9e-4752-aaac-f3c08ef5f966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:03.891094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07dcf9-5e9e-4752-aaac-f3c08ef5f966 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:03.891380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07dcf9-5e9e-4752-aaac-f3c08ef5f966 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:03.904552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07dcf9-5e9e-4752-aaac-f3c08ef5f966 admin admin] PATH_INFO: `/v3/projects/1f20b8bbe3d2429a8ab4d8271d7a431b/users/de7fca356017414caa0b1652fdccf7e3/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:03.929038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07dcf9-5e9e-4752-aaac-f3c08ef5f966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f20b8bbe3d2429a8ab4d8271d7a431b, user_id=de7fca356017414caa0b1652fdccf7e3, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:03.958699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07dcf9-5e9e-4752-aaac-f3c08ef5f966 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:03.997481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1141/2270] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:03 2022] PUT /identity/v3/projects/1f20b8bbe3d2429a8ab4d8271d7a431b/users/de7fca356017414caa0b1652fdccf7e3/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:04.018622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d4375-2677-4d41-afe8-1cd7cf449df3 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:04.019270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d4375-2677-4d41-afe8-1cd7cf449df3 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:04.019606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d4375-2677-4d41-afe8-1cd7cf449df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:04.119016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b64d4375-2677-4d41-afe8-1cd7cf449df3 None None] MFA Rules not processed for user `de7fca356017414caa0b1652fdccf7e3`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:04.122102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b64d4375-2677-4d41-afe8-1cd7cf449df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:04.475864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1130/2271] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:04.477781 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07dcf9-5e9e-4752-aaac-f3c08ef5f966 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:04.797969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07dcf9-5e9e-4752-aaac-f3c08ef5f966 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:04.808519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7d058-a2ee-4e42-ba00-c02609d63e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:04.810033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7d058-a2ee-4e42-ba00-c02609d63e94 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:04.810950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7d058-a2ee-4e42-ba00-c02609d63e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:04.811453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7d058-a2ee-4e42-ba00-c02609d63e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:04.813208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7d058-a2ee-4e42-ba00-c02609d63e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:04.828199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7d058-a2ee-4e42-ba00-c02609d63e94 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:04.945626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1142/2272] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:04 2022] POST /identity/v3/projects => generated 381 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:04.951703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d4375-2677-4d41-afe8-1cd7cf449df3 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:04.978789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d4375-2677-4d41-afe8-1cd7cf449df3 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:05.107279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654eaa6-a435-41c5-a5dd-04a9c558a8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:05.108498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654eaa6-a435-41c5-a5dd-04a9c558a8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:05.108985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654eaa6-a435-41c5-a5dd-04a9c558a8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:05.109491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654eaa6-a435-41c5-a5dd-04a9c558a8c1 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:05.111055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654eaa6-a435-41c5-a5dd-04a9c558a8c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:05.115105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654eaa6-a435-41c5-a5dd-04a9c558a8c1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:05.182805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1131/2273] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:04 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:05.188508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7d058-a2ee-4e42-ba00-c02609d63e94 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:05.222438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7d058-a2ee-4e42-ba00-c02609d63e94 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:05.254423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f6631-7269-448b-bff7-cb9672350a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:05.257349 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f6631-7269-448b-bff7-cb9672350a94 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:05.257929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f6631-7269-448b-bff7-cb9672350a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:05.258428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f6631-7269-448b-bff7-cb9672350a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:05.276116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f6631-7269-448b-bff7-cb9672350a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:05.280904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f6631-7269-448b-bff7-cb9672350a94 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:05.303839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1143/2274] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:05.304566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654eaa6-a435-41c5-a5dd-04a9c558a8c1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:05.317585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654eaa6-a435-41c5-a5dd-04a9c558a8c1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:05.323080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814106d9-d27d-45f3-8fdd-57b879965ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:05.324774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814106d9-d27d-45f3-8fdd-57b879965ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:05.325544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814106d9-d27d-45f3-8fdd-57b879965ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:05.326001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814106d9-d27d-45f3-8fdd-57b879965ad1 admin admin] PATH_INFO: `/v3/projects/31965219e7ba4d52a25616bf6f9ca680/users/9bdcf2db96334c92872f7ee92274a2e9/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:05.356476 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814106d9-d27d-45f3-8fdd-57b879965ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31965219e7ba4d52a25616bf6f9ca680, user_id=9bdcf2db96334c92872f7ee92274a2e9, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:05.362440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814106d9-d27d-45f3-8fdd-57b879965ad1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:05.382121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1132/2275] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:05 2022] PUT /identity/v3/projects/31965219e7ba4d52a25616bf6f9ca680/users/9bdcf2db96334c92872f7ee92274a2e9/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:05.388380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f6631-7269-448b-bff7-cb9672350a94 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:05.400456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f6631-7269-448b-bff7-cb9672350a94 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:05.520753 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a9b93-27fd-4746-9e32-afed05fb7ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:05.523491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9b93-27fd-4746-9e32-afed05fb7ded admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:05.523966 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9b93-27fd-4746-9e32-afed05fb7ded admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:05.524478 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9b93-27fd-4746-9e32-afed05fb7ded admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:05.525307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a9b93-27fd-4746-9e32-afed05fb7ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:05.528483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a9b93-27fd-4746-9e32-afed05fb7ded admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:05.546286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1144/2276] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:05.551966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814106d9-d27d-45f3-8fdd-57b879965ad1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:05.565058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814106d9-d27d-45f3-8fdd-57b879965ad1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:05.570942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26e1dc-6f61-471a-a7a5-eed59c018e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:05.572255 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e1dc-6f61-471a-a7a5-eed59c018e5a admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:05.572835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e1dc-6f61-471a-a7a5-eed59c018e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:05.573352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26e1dc-6f61-471a-a7a5-eed59c018e5a admin admin] PATH_INFO: `/v3/projects/31965219e7ba4d52a25616bf6f9ca680/users/9bdcf2db96334c92872f7ee92274a2e9/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:05.578093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26e1dc-6f61-471a-a7a5-eed59c018e5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=31965219e7ba4d52a25616bf6f9ca680, user_id=9bdcf2db96334c92872f7ee92274a2e9, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:05.582765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26e1dc-6f61-471a-a7a5-eed59c018e5a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:05.600441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1133/2277] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:05 2022] PUT /identity/v3/projects/31965219e7ba4d52a25616bf6f9ca680/users/9bdcf2db96334c92872f7ee92274a2e9/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:05.619464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad201b7-5ad1-4114-b731-cfdb341e26ba None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:05.619967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad201b7-5ad1-4114-b731-cfdb341e26ba None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:05.620460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad201b7-5ad1-4114-b731-cfdb341e26ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:05.676923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2ad201b7-5ad1-4114-b731-cfdb341e26ba None None] MFA Rules not processed for user `9bdcf2db96334c92872f7ee92274a2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:05.678243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2ad201b7-5ad1-4114-b731-cfdb341e26ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:05.805722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1145/2278] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:05.825469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26e1dc-6f61-471a-a7a5-eed59c018e5a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:05.828102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1d26e1dc-6f61-471a-a7a5-eed59c018e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:05.989952 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26e1dc-6f61-471a-a7a5-eed59c018e5a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:06.007290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccfe45-101c-4d54-a0ed-ce4ef2b16ffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:06.007290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccfe45-101c-4d54-a0ed-ce4ef2b16ffa service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:06.007290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccfe45-101c-4d54-a0ed-ce4ef2b16ffa service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:06.007290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccfe45-101c-4d54-a0ed-ce4ef2b16ffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:06.031776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccfe45-101c-4d54-a0ed-ce4ef2b16ffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:06.034625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccfe45-101c-4d54-a0ed-ce4ef2b16ffa service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:06.053268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1134/2279] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:06.170442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad201b7-5ad1-4114-b731-cfdb341e26ba None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:06.189463 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad201b7-5ad1-4114-b731-cfdb341e26ba None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:06.196028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccebb7c-f5cd-426b-8bd1-9b498a4ce771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:06.197380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccebb7c-f5cd-426b-8bd1-9b498a4ce771 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:06.197885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccebb7c-f5cd-426b-8bd1-9b498a4ce771 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:06.198249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccebb7c-f5cd-426b-8bd1-9b498a4ce771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:06.323035 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccebb7c-f5cd-426b-8bd1-9b498a4ce771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:06.324659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccebb7c-f5cd-426b-8bd1-9b498a4ce771 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:06.339169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1146/2280] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:17.415589 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccfe45-101c-4d54-a0ed-ce4ef2b16ffa service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:17.619977 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccfe45-101c-4d54-a0ed-ce4ef2b16ffa service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:17.629213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:17.630025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:17.632760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:17.633337 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] PATH_INFO: `/v3/users/de7fca356017414caa0b1652fdccf7e3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:17.635638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7fca356017414caa0b1652fdccf7e3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:17.637778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:17.736171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7fca356017414caa0b1652fdccf7e3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:17.737205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7fca356017414caa0b1652fdccf7e3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:17.766896 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7fca356017414caa0b1652fdccf7e3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:17.767650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7fca356017414caa0b1652fdccf7e3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:17.778921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7fca356017414caa0b1652fdccf7e3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:17.807550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1135/2281] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:17 2022] DELETE /identity/v3/users/de7fca356017414caa0b1652fdccf7e3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:17.813885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae2aae-9c91-4e80-9853-f6dea84004e8 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:17.814509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae2aae-9c91-4e80-9853-f6dea84004e8 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:17.814509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae2aae-9c91-4e80-9853-f6dea84004e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:17.853076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-19ae2aae-9c91-4e80-9853-f6dea84004e8 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:17.854341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-19ae2aae-9c91-4e80-9853-f6dea84004e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:17.996728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1147/2282] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:18.016853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:18.020131 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:18.099055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b8c61-59af-4653-868e-d1493709480c admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:18.105899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00257b-142a-4d10-9d6b-3db3810a25b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:18.107523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00257b-142a-4d10-9d6b-3db3810a25b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:18.107911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00257b-142a-4d10-9d6b-3db3810a25b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:18.108271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00257b-142a-4d10-9d6b-3db3810a25b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:18.123064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00257b-142a-4d10-9d6b-3db3810a25b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:18.125361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00257b-142a-4d10-9d6b-3db3810a25b2 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:18.139113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1136/2283] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:18.508464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae2aae-9c91-4e80-9853-f6dea84004e8 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:18.509151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-19ae2aae-9c91-4e80-9853-f6dea84004e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:18.522581 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae2aae-9c91-4e80-9853-f6dea84004e8 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:18.528889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:18.530149 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:18.530149 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:18.530456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] PATH_INFO: `/v3/projects/1f20b8bbe3d2429a8ab4d8271d7a431b` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:18.532183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f20b8bbe3d2429a8ab4d8271d7a431b)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:18.534152 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:18.584593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f20b8bbe3d2429a8ab4d8271d7a431b', 'request_id': 'req-b62a12af-a4f0-4719-907a-790edee62b79'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:18.590161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f20b8bbe3d2429a8ab4d8271d7a431b', 'request_id': 'req-b62a12af-a4f0-4719-907a-790edee62b79'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:18.599425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f20b8bbe3d2429a8ab4d8271d7a431b', 'request_id': 'req-b62a12af-a4f0-4719-907a-790edee62b79'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:18.611107 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] The token cache is being invalidate because project 1f20b8bbe3d2429a8ab4d8271d7a431b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:18.611388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:18.612058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:18.613630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1148/2284] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:18 2022] DELETE /identity/v3/projects/1f20b8bbe3d2429a8ab4d8271d7a431b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:18.618034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00257b-142a-4d10-9d6b-3db3810a25b2 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:18.628927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00257b-142a-4d10-9d6b-3db3810a25b2 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:18.689401 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:18.690314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:18.690539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:18.690836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] PATH_INFO: `/v3/users/9bdcf2db96334c92872f7ee92274a2e9` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:18.692072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bdcf2db96334c92872f7ee92274a2e9)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:18.694092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:18.751899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bdcf2db96334c92872f7ee92274a2e9'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:18.753075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bdcf2db96334c92872f7ee92274a2e9'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:18.769374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bdcf2db96334c92872f7ee92274a2e9'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:18.770411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bdcf2db96334c92872f7ee92274a2e9'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:18.779206 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ca9979fe-0aaa-4471-a52a-ae4a8f2707f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bdcf2db96334c92872f7ee92274a2e9'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:18.800065 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1137/2285] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:18 2022] DELETE /identity/v3/users/9bdcf2db96334c92872f7ee92274a2e9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:19.138871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:19.141153 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:19.156202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a12af-a4f0-4719-907a-790edee62b79 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:19.234246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d6fb3-fd87-417c-9d13-27311e798a69 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.234549 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d6fb3-fd87-417c-9d13-27311e798a69 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.234861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d6fb3-fd87-417c-9d13-27311e798a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.253662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:19.254549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.254987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.255263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] PATH_INFO: `/v3/projects/31965219e7ba4d52a25616bf6f9ca680` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.255544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-da9d6fb3-fd87-417c-9d13-27311e798a69 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:19.256813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31965219e7ba4d52a25616bf6f9ca680)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:19.257320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-da9d6fb3-fd87-417c-9d13-27311e798a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:19.258774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:19.316110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31965219e7ba4d52a25616bf6f9ca680', 'request_id': 'req-c5817000-e8ed-4284-b657-315763669a07'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:19.328714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31965219e7ba4d52a25616bf6f9ca680', 'request_id': 'req-c5817000-e8ed-4284-b657-315763669a07'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:19.337468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31965219e7ba4d52a25616bf6f9ca680', 'request_id': 'req-c5817000-e8ed-4284-b657-315763669a07'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:19.354157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1138/2286] 10.222.0.79 () {58 vars in 967 bytes} [Fri May 13 14:03:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:19.356979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] The token cache is being invalidate because project 31965219e7ba4d52a25616bf6f9ca680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:19.357367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:19.358162 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:19.360487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1149/2287] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:19 2022] DELETE /identity/v3/projects/31965219e7ba4d52a25616bf6f9ca680 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:19.365922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d6907-8d46-476c-aea4-bafeb09a9a74 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.366378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d6907-8d46-476c-aea4-bafeb09a9a74 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.366796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d6907-8d46-476c-aea4-bafeb09a9a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.371394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:19.373864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:19.391269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-144d6907-8d46-476c-aea4-bafeb09a9a74 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:19.392412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-144d6907-8d46-476c-aea4-bafeb09a9a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:19.468993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5817000-e8ed-4284-b657-315763669a07 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:19.479458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1139/2288] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:19.481363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a100bb-1a04-4254-b4e3-67336dfc5bd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:19.482133 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a100bb-1a04-4254-b4e3-67336dfc5bd5 service swift] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.482517 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a100bb-1a04-4254-b4e3-67336dfc5bd5 service swift] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.482909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a100bb-1a04-4254-b4e3-67336dfc5bd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.485763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c1a100bb-1a04-4254-b4e3-67336dfc5bd5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:19.488487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1cf65-db35-4f09-81b9-5b1a8a8d6bbc None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.488927 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1cf65-db35-4f09-81b9-5b1a8a8d6bbc None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.489296 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1cf65-db35-4f09-81b9-5b1a8a8d6bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.518576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e3c1cf65-db35-4f09-81b9-5b1a8a8d6bbc None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:19.520768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e3c1cf65-db35-4f09-81b9-5b1a8a8d6bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:19.533760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1140/2289] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:19.543827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1cf65-db35-4f09-81b9-5b1a8a8d6bbc None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:19.554650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1cf65-db35-4f09-81b9-5b1a8a8d6bbc None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:19.559403 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb4f67-c9b7-44b4-be78-44cbedca96dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:19.560424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb4f67-c9b7-44b4-be78-44cbedca96dd admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.560424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb4f67-c9b7-44b4-be78-44cbedca96dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.560667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb4f67-c9b7-44b4-be78-44cbedca96dd admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.561320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb4f67-c9b7-44b4-be78-44cbedca96dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:19.561665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb4f67-c9b7-44b4-be78-44cbedca96dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:19.562339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a100bb-1a04-4254-b4e3-67336dfc5bd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:19.563338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb4f67-c9b7-44b4-be78-44cbedca96dd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:19.563928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a100bb-1a04-4254-b4e3-67336dfc5bd5 service swift] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:19.572669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1150/2290] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:03:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 14:03:19.577254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1141/2291] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 14:03:19.581402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a100bb-1a04-4254-b4e3-67336dfc5bd5 service swift] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:19.595236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a100bb-1a04-4254-b4e3-67336dfc5bd5 service swift] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:19.601821 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe1d7f-c942-46b3-a791-c55db8c2f8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:19.603112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe1d7f-c942-46b3-a791-c55db8c2f8e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.603607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe1d7f-c942-46b3-a791-c55db8c2f8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.604092 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe1d7f-c942-46b3-a791-c55db8c2f8e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.605410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe1d7f-c942-46b3-a791-c55db8c2f8e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:19.609482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe1d7f-c942-46b3-a791-c55db8c2f8e9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:19.643208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1151/2292] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:19 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:19.647586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb4f67-c9b7-44b4-be78-44cbedca96dd admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:19.661049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb4f67-c9b7-44b4-be78-44cbedca96dd admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:19.730155 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf6b6f-e670-45d9-b70b-6b2d1b6844bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:19.730940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf6b6f-e670-45d9-b70b-6b2d1b6844bb admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.731224 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf6b6f-e670-45d9-b70b-6b2d1b6844bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.731493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf6b6f-e670-45d9-b70b-6b2d1b6844bb admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.732369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf6b6f-e670-45d9-b70b-6b2d1b6844bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:19.735077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf6b6f-e670-45d9-b70b-6b2d1b6844bb admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:19.758449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1142/2293] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:19 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:19.762214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe1d7f-c942-46b3-a791-c55db8c2f8e9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:19.773917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe1d7f-c942-46b3-a791-c55db8c2f8e9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:19.781149 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8429b-ee42-4546-8faf-f1e95f5fa975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:19.781832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8429b-ee42-4546-8faf-f1e95f5fa975 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.782220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8429b-ee42-4546-8faf-f1e95f5fa975 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.782372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8429b-ee42-4546-8faf-f1e95f5fa975 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.783179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8429b-ee42-4546-8faf-f1e95f5fa975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:19.784863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8429b-ee42-4546-8faf-f1e95f5fa975 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:19.797358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1152/2294] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:19.804685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf6b6f-e670-45d9-b70b-6b2d1b6844bb admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:19.823526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf6b6f-e670-45d9-b70b-6b2d1b6844bb admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:19.829020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed4001-cebf-48b8-b5dd-6115cedd740d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:19.829694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed4001-cebf-48b8-b5dd-6115cedd740d admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.829985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed4001-cebf-48b8-b5dd-6115cedd740d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.830258 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed4001-cebf-48b8-b5dd-6115cedd740d admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.831032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed4001-cebf-48b8-b5dd-6115cedd740d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:19.832856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed4001-cebf-48b8-b5dd-6115cedd740d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:19.844126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1143/2295] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:19.848679 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8429b-ee42-4546-8faf-f1e95f5fa975 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:19.861059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8429b-ee42-4546-8faf-f1e95f5fa975 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:19.866126 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012153d4-3f52-440d-a072-f7bf0fd1c307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:19.867059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012153d4-3f52-440d-a072-f7bf0fd1c307 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.867339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012153d4-3f52-440d-a072-f7bf0fd1c307 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.867607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012153d4-3f52-440d-a072-f7bf0fd1c307 admin admin] PATH_INFO: `/v3/projects/c5a6881b2b9745c9b23da556eb6d3df7/users/95cc77fa1e704440af99f845678b1033/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.884614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012153d4-3f52-440d-a072-f7bf0fd1c307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a6881b2b9745c9b23da556eb6d3df7, user_id=95cc77fa1e704440af99f845678b1033, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:19.887380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012153d4-3f52-440d-a072-f7bf0fd1c307 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:19.902359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1153/2296] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:19 2022] PUT /identity/v3/projects/c5a6881b2b9745c9b23da556eb6d3df7/users/95cc77fa1e704440af99f845678b1033/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:19.915287 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b9149-8dbf-4726-833d-06d55bc4c32f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:19.915568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b9149-8dbf-4726-833d-06d55bc4c32f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:19.915897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b9149-8dbf-4726-833d-06d55bc4c32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:19.947446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7c0b9149-8dbf-4726-833d-06d55bc4c32f None None] MFA Rules not processed for user `95cc77fa1e704440af99f845678b1033`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:19.949030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7c0b9149-8dbf-4726-833d-06d55bc4c32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:20.035370 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1144/2297] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:20.039868 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012153d4-3f52-440d-a072-f7bf0fd1c307 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:20.111533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012153d4-3f52-440d-a072-f7bf0fd1c307 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:20.116844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c4ed5-b795-4418-bc73-1de151152fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:20.117534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c4ed5-b795-4418-bc73-1de151152fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:20.117815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c4ed5-b795-4418-bc73-1de151152fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:20.118093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c4ed5-b795-4418-bc73-1de151152fd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:20.118874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c4ed5-b795-4418-bc73-1de151152fd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:20.120827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c4ed5-b795-4418-bc73-1de151152fd8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:20.149962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1154/2298] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:20 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:20.154285 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b9149-8dbf-4726-833d-06d55bc4c32f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:20.165960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b9149-8dbf-4726-833d-06d55bc4c32f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:20.226772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28774e15-06c7-466e-b413-54a5ef5e24fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:20.227491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28774e15-06c7-466e-b413-54a5ef5e24fe admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:20.227775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28774e15-06c7-466e-b413-54a5ef5e24fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:20.228050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28774e15-06c7-466e-b413-54a5ef5e24fe admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:20.228746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28774e15-06c7-466e-b413-54a5ef5e24fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:20.230718 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28774e15-06c7-466e-b413-54a5ef5e24fe admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:20.254826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1145/2299] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:20 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:20.259184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c4ed5-b795-4418-bc73-1de151152fd8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:20.270368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c4ed5-b795-4418-bc73-1de151152fd8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:20.275205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbfd24-d6d4-4e16-b5cb-3ee0a92484c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:20.275869 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbfd24-d6d4-4e16-b5cb-3ee0a92484c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:20.276150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbfd24-d6d4-4e16-b5cb-3ee0a92484c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:20.276505 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbfd24-d6d4-4e16-b5cb-3ee0a92484c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:20.277177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbfd24-d6d4-4e16-b5cb-3ee0a92484c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:20.278815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbfd24-d6d4-4e16-b5cb-3ee0a92484c6 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:20.290560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1155/2300] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:20.294820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28774e15-06c7-466e-b413-54a5ef5e24fe admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:20.306833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28774e15-06c7-466e-b413-54a5ef5e24fe admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:20.313648 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b8f4f-ed4d-4feb-b861-05155fd7a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:20.314321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b8f4f-ed4d-4feb-b861-05155fd7a998 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:20.314595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b8f4f-ed4d-4feb-b861-05155fd7a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:20.314919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b8f4f-ed4d-4feb-b861-05155fd7a998 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:20.315641 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b8f4f-ed4d-4feb-b861-05155fd7a998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:20.317469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b8f4f-ed4d-4feb-b861-05155fd7a998 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:20.329912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1146/2301] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:20.335536 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbfd24-d6d4-4e16-b5cb-3ee0a92484c6 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:20.346002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbfd24-d6d4-4e16-b5cb-3ee0a92484c6 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:20.351013 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b583bc-9361-4fe7-9694-e6b096884049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:20.351916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b583bc-9361-4fe7-9694-e6b096884049 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:20.352189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b583bc-9361-4fe7-9694-e6b096884049 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:20.352464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b583bc-9361-4fe7-9694-e6b096884049 admin admin] PATH_INFO: `/v3/projects/34086cd45b194ba88a091f821b87119c/users/1123a5f3225e42109104bd8093a5b655/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:20.369844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b583bc-9361-4fe7-9694-e6b096884049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34086cd45b194ba88a091f821b87119c, user_id=1123a5f3225e42109104bd8093a5b655, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:20.373156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b583bc-9361-4fe7-9694-e6b096884049 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:20.390502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1156/2302] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:20 2022] PUT /identity/v3/projects/34086cd45b194ba88a091f821b87119c/users/1123a5f3225e42109104bd8093a5b655/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:20.403450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17b609-d6ec-4ea3-8dee-e518baba564a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:20.403909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17b609-d6ec-4ea3-8dee-e518baba564a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:20.404402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17b609-d6ec-4ea3-8dee-e518baba564a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:20.435680 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1f17b609-d6ec-4ea3-8dee-e518baba564a None None] MFA Rules not processed for user `1123a5f3225e42109104bd8093a5b655`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:20.438485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1f17b609-d6ec-4ea3-8dee-e518baba564a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:20.555616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1147/2303] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:20.563564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b583bc-9361-4fe7-9694-e6b096884049 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:20.659798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b583bc-9361-4fe7-9694-e6b096884049 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:20.664660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bc225-a7c8-4ce3-926e-f60a7710a3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:20.665329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bc225-a7c8-4ce3-926e-f60a7710a3fe admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:20.666112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bc225-a7c8-4ce3-926e-f60a7710a3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:20.666112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bc225-a7c8-4ce3-926e-f60a7710a3fe admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:20.666583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bc225-a7c8-4ce3-926e-f60a7710a3fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:20.668583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bc225-a7c8-4ce3-926e-f60a7710a3fe admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:20.704345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1157/2304] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:20 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:20.707591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17b609-d6ec-4ea3-8dee-e518baba564a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:20.718938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17b609-d6ec-4ea3-8dee-e518baba564a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:20.783326 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396071c0-1742-4dac-96e9-228bcd77f9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:20.784062 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396071c0-1742-4dac-96e9-228bcd77f9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:20.784380 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396071c0-1742-4dac-96e9-228bcd77f9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:20.784710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396071c0-1742-4dac-96e9-228bcd77f9f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:20.785478 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396071c0-1742-4dac-96e9-228bcd77f9f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:20.787692 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396071c0-1742-4dac-96e9-228bcd77f9f0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:20.813344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1148/2305] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:20 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:20.817915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bc225-a7c8-4ce3-926e-f60a7710a3fe admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:20.830780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bc225-a7c8-4ce3-926e-f60a7710a3fe admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:20.836563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499312a-a20a-46c4-b16e-82662c4a74b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:20.837460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499312a-a20a-46c4-b16e-82662c4a74b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:20.837836 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499312a-a20a-46c4-b16e-82662c4a74b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:20.838331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499312a-a20a-46c4-b16e-82662c4a74b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:20.839542 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499312a-a20a-46c4-b16e-82662c4a74b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:20.841210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499312a-a20a-46c4-b16e-82662c4a74b1 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:20.854409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1158/2306] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:20.859913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396071c0-1742-4dac-96e9-228bcd77f9f0 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:20.871002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396071c0-1742-4dac-96e9-228bcd77f9f0 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:20.876753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490cfc2-7ded-47cc-aa2a-d9b43416a25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:20.877779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490cfc2-7ded-47cc-aa2a-d9b43416a25d admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:20.878100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490cfc2-7ded-47cc-aa2a-d9b43416a25d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:20.878467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490cfc2-7ded-47cc-aa2a-d9b43416a25d admin admin] PATH_INFO: `/v3/projects/9b71e55fa0064c93bfebc188f15d8403/users/ddb99f89278140c5b46fa049d570fbfe/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:20.895329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490cfc2-7ded-47cc-aa2a-d9b43416a25d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b71e55fa0064c93bfebc188f15d8403, user_id=ddb99f89278140c5b46fa049d570fbfe, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:20.898250 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490cfc2-7ded-47cc-aa2a-d9b43416a25d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:20.918236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1149/2307] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:20 2022] PUT /identity/v3/projects/9b71e55fa0064c93bfebc188f15d8403/users/ddb99f89278140c5b46fa049d570fbfe/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:20.923614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499312a-a20a-46c4-b16e-82662c4a74b1 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:20.935064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499312a-a20a-46c4-b16e-82662c4a74b1 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:20.998991 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f75d0-adf0-4413-867b-a51abb9923fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:20.999717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f75d0-adf0-4413-867b-a51abb9923fa admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:21.000071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f75d0-adf0-4413-867b-a51abb9923fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:21.000365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f75d0-adf0-4413-867b-a51abb9923fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:21.001093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f75d0-adf0-4413-867b-a51abb9923fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:21.002700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f75d0-adf0-4413-867b-a51abb9923fa admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:21.014287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1159/2308] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:03:21.019316 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490cfc2-7ded-47cc-aa2a-d9b43416a25d admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:21.031457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490cfc2-7ded-47cc-aa2a-d9b43416a25d admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:21.036328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd0b70-01c8-481b-9a5a-e807bf3b8623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:21.037232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd0b70-01c8-481b-9a5a-e807bf3b8623 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:21.037503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd0b70-01c8-481b-9a5a-e807bf3b8623 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:21.037829 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd0b70-01c8-481b-9a5a-e807bf3b8623 admin admin] PATH_INFO: `/v3/projects/9b71e55fa0064c93bfebc188f15d8403/users/ddb99f89278140c5b46fa049d570fbfe/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:21.039881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd0b70-01c8-481b-9a5a-e807bf3b8623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b71e55fa0064c93bfebc188f15d8403, user_id=ddb99f89278140c5b46fa049d570fbfe, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:21.042569 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd0b70-01c8-481b-9a5a-e807bf3b8623 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:21.054613 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1150/2309] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:21 2022] PUT /identity/v3/projects/9b71e55fa0064c93bfebc188f15d8403/users/ddb99f89278140c5b46fa049d570fbfe/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:21.066124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0f930-2e30-45bf-a72e-3c23bd9c6e39 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:21.066407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0f930-2e30-45bf-a72e-3c23bd9c6e39 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:21.066681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0f930-2e30-45bf-a72e-3c23bd9c6e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:21.097173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6af0f930-2e30-45bf-a72e-3c23bd9c6e39 None None] MFA Rules not processed for user `ddb99f89278140c5b46fa049d570fbfe`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:21.098473 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6af0f930-2e30-45bf-a72e-3c23bd9c6e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:21.178322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1160/2310] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:21.190339 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd0b70-01c8-481b-9a5a-e807bf3b8623 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:21.192688 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-afcd0b70-01c8-481b-9a5a-e807bf3b8623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:21.256932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd0b70-01c8-481b-9a5a-e807bf3b8623 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:21.260902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53ed33-5202-4401-b7d4-7df18ce6bfae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:21.261560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53ed33-5202-4401-b7d4-7df18ce6bfae service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:21.261875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53ed33-5202-4401-b7d4-7df18ce6bfae service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:21.262172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53ed33-5202-4401-b7d4-7df18ce6bfae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:21.272145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53ed33-5202-4401-b7d4-7df18ce6bfae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:21.273695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53ed33-5202-4401-b7d4-7df18ce6bfae service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:21.284546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1151/2311] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:21.873758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0f930-2e30-45bf-a72e-3c23bd9c6e39 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:21.875833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6af0f930-2e30-45bf-a72e-3c23bd9c6e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:21.888828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0f930-2e30-45bf-a72e-3c23bd9c6e39 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:21.897225 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837484d1-de76-4a39-9575-31bed47aaecf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:21.897899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837484d1-de76-4a39-9575-31bed47aaecf service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:21.898184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837484d1-de76-4a39-9575-31bed47aaecf service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:21.898486 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837484d1-de76-4a39-9575-31bed47aaecf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:21.965369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837484d1-de76-4a39-9575-31bed47aaecf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:21.967072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837484d1-de76-4a39-9575-31bed47aaecf service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:21.979477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1161/2312] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:26.142701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53ed33-5202-4401-b7d4-7df18ce6bfae service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:26.151430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53ed33-5202-4401-b7d4-7df18ce6bfae service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:26.156111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935ed20-1149-46fa-aaec-bd80afe3594e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:26.156111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935ed20-1149-46fa-aaec-bd80afe3594e service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:26.156111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935ed20-1149-46fa-aaec-bd80afe3594e service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:26.156513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935ed20-1149-46fa-aaec-bd80afe3594e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:26.215389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935ed20-1149-46fa-aaec-bd80afe3594e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:26.217034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935ed20-1149-46fa-aaec-bd80afe3594e service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:26.234196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1152/2313] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:28.506351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837484d1-de76-4a39-9575-31bed47aaecf service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:28.507843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-837484d1-de76-4a39-9575-31bed47aaecf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:28.576242 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837484d1-de76-4a39-9575-31bed47aaecf service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:28.581008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07519a42-b849-4f62-a358-83cf8e6e0268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:28.581947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07519a42-b849-4f62-a358-83cf8e6e0268 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:28.582116 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07519a42-b849-4f62-a358-83cf8e6e0268 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:28.582389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07519a42-b849-4f62-a358-83cf8e6e0268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:28.584817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-07519a42-b849-4f62-a358-83cf8e6e0268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:28.626081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935ed20-1149-46fa-aaec-bd80afe3594e service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:28.666570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07519a42-b849-4f62-a358-83cf8e6e0268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:28.668297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07519a42-b849-4f62-a358-83cf8e6e0268 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:28.691463 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1162/2314] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:28.707351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935ed20-1149-46fa-aaec-bd80afe3594e service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:28.714035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:28.715279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:28.715563 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:28.715849 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] PATH_INFO: `/v3/users/95cc77fa1e704440af99f845678b1033` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:28.717217 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] RBAC: Authorizing `identity:delete_user(user_id=95cc77fa1e704440af99f845678b1033)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:28.719378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:28.787572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95cc77fa1e704440af99f845678b1033'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:28.788219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95cc77fa1e704440af99f845678b1033'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:28.802942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95cc77fa1e704440af99f845678b1033'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:28.803770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95cc77fa1e704440af99f845678b1033'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:28.811373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95cc77fa1e704440af99f845678b1033'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:28.830483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1153/2315] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:28 2022] DELETE /identity/v3/users/95cc77fa1e704440af99f845678b1033 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:28.833201 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0d22b-0fdc-4643-bd54-c4f0506089cf None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:28.833533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0d22b-0fdc-4643-bd54-c4f0506089cf None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:28.833819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0d22b-0fdc-4643-bd54-c4f0506089cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:28.854446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1ec0d22b-0fdc-4643-bd54-c4f0506089cf None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:28.855797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1ec0d22b-0fdc-4643-bd54-c4f0506089cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:28.918757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1163/2316] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:28.932429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:28.934020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:28.998474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccff85-1983-40ad-8006-ffe71faccafe admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:29.003355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c285c6-2dce-46c5-bdb6-a667c6ff7eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:29.004007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c285c6-2dce-46c5-bdb6-a667c6ff7eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:29.004291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c285c6-2dce-46c5-bdb6-a667c6ff7eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:29.004571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c285c6-2dce-46c5-bdb6-a667c6ff7eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:29.015473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c285c6-2dce-46c5-bdb6-a667c6ff7eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:29.019909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c285c6-2dce-46c5-bdb6-a667c6ff7eb9 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:29.030942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1154/2317] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:29.498543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0d22b-0fdc-4643-bd54-c4f0506089cf None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:29.500537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1ec0d22b-0fdc-4643-bd54-c4f0506089cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:29.517161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0d22b-0fdc-4643-bd54-c4f0506089cf None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:29.523544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:29.525493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:29.525831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:29.525978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] PATH_INFO: `/v3/projects/c5a6881b2b9745c9b23da556eb6d3df7` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:29.527824 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5a6881b2b9745c9b23da556eb6d3df7)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:29.530033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:29.580149 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5a6881b2b9745c9b23da556eb6d3df7', 'request_id': 'req-d5126352-20d7-4d3c-bfb9-994e3e97b795'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:29.589599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5a6881b2b9745c9b23da556eb6d3df7', 'request_id': 'req-d5126352-20d7-4d3c-bfb9-994e3e97b795'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:29.599241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5a6881b2b9745c9b23da556eb6d3df7', 'request_id': 'req-d5126352-20d7-4d3c-bfb9-994e3e97b795'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:29.610637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] The token cache is being invalidate because project c5a6881b2b9745c9b23da556eb6d3df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:29.610958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:29.611614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:29.613168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1164/2318] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:29 2022] DELETE /identity/v3/projects/c5a6881b2b9745c9b23da556eb6d3df7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:29.618332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c285c6-2dce-46c5-bdb6-a667c6ff7eb9 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:29.629684 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c285c6-2dce-46c5-bdb6-a667c6ff7eb9 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:29.696583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:29.697497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:29.697779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:29.698095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] PATH_INFO: `/v3/users/1123a5f3225e42109104bd8093a5b655` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:29.699348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1123a5f3225e42109104bd8093a5b655)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:29.701294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:29.756483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1123a5f3225e42109104bd8093a5b655'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:29.757246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1123a5f3225e42109104bd8093a5b655'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:29.770272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1123a5f3225e42109104bd8093a5b655'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:29.770924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1123a5f3225e42109104bd8093a5b655'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:29.777567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1123a5f3225e42109104bd8093a5b655'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:29.798833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1155/2319] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:29 2022] DELETE /identity/v3/users/1123a5f3225e42109104bd8093a5b655 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:30.215916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:30.218414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:30.245009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5126352-20d7-4d3c-bfb9-994e3e97b795 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:30.311747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:30.312904 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:30.313194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:30.313546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] PATH_INFO: `/v3/projects/34086cd45b194ba88a091f821b87119c` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:30.314931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] RBAC: Authorizing `identity:delete_project(project_id=34086cd45b194ba88a091f821b87119c)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:30.317085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:30.367378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34086cd45b194ba88a091f821b87119c', 'request_id': 'req-ef981327-b2a2-43ab-a165-6ffbef45b24d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:30.373938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34086cd45b194ba88a091f821b87119c', 'request_id': 'req-ef981327-b2a2-43ab-a165-6ffbef45b24d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:30.382048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34086cd45b194ba88a091f821b87119c', 'request_id': 'req-ef981327-b2a2-43ab-a165-6ffbef45b24d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:30.394090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] The token cache is being invalidate because project 34086cd45b194ba88a091f821b87119c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:30.394362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:30.395111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:30.400052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1165/2320] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:30 2022] DELETE /identity/v3/projects/34086cd45b194ba88a091f821b87119c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:30.401164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:30.402907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:30.423912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130cc9e-c442-48b5-91fb-ecb2f6c22fdb admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:30.482314 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:30.483246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:30.483536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:30.483810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] PATH_INFO: `/v3/users/ddb99f89278140c5b46fa049d570fbfe` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:30.485017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddb99f89278140c5b46fa049d570fbfe)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:30.487033 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:30.536001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddb99f89278140c5b46fa049d570fbfe'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:30.536829 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddb99f89278140c5b46fa049d570fbfe'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:30.548982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddb99f89278140c5b46fa049d570fbfe'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:30.549654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddb99f89278140c5b46fa049d570fbfe'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:30.556488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6e3aacd9-f3c1-4955-9042-77f87be09373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddb99f89278140c5b46fa049d570fbfe'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:30.570555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1156/2321] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:30 2022] DELETE /identity/v3/users/ddb99f89278140c5b46fa049d570fbfe => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:30.920839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:30.923114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:30.939143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef981327-b2a2-43ab-a165-6ffbef45b24d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:31.002203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:31.003205 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.003559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.003909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] PATH_INFO: `/v3/projects/9b71e55fa0064c93bfebc188f15d8403` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.005215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b71e55fa0064c93bfebc188f15d8403)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:31.007257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:31.058361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b71e55fa0064c93bfebc188f15d8403', 'request_id': 'req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:31.065125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b71e55fa0064c93bfebc188f15d8403', 'request_id': 'req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:31.075315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b71e55fa0064c93bfebc188f15d8403', 'request_id': 'req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:31.089061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] The token cache is being invalidate because project 9b71e55fa0064c93bfebc188f15d8403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:31.089713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:31.090549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b4be88b6-bfc9-49c5-8dc2-c8c01ba84c15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:31.092356 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1166/2322] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:30 2022] DELETE /identity/v3/projects/9b71e55fa0064c93bfebc188f15d8403 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:31.100011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30f966-403f-498f-83cc-6da2409852a7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.101398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30f966-403f-498f-83cc-6da2409852a7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.101895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30f966-403f-498f-83cc-6da2409852a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.126401 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-3e30f966-403f-498f-83cc-6da2409852a7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:31.129388 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-3e30f966-403f-498f-83cc-6da2409852a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:31.237192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1157/2323] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:31.247433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2712d7b-15e4-4927-97ac-860f82b1b8b7 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.247721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2712d7b-15e4-4927-97ac-860f82b1b8b7 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.248167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2712d7b-15e4-4927-97ac-860f82b1b8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.271372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-d2712d7b-15e4-4927-97ac-860f82b1b8b7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:31.272556 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d2712d7b-15e4-4927-97ac-860f82b1b8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:31.286154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1167/2324] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:31.290724 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30f966-403f-498f-83cc-6da2409852a7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:31.312184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30f966-403f-498f-83cc-6da2409852a7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:31.326541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeeb125-22a9-4271-ade6-70128537ab47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:31.331665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeb125-22a9-4271-ade6-70128537ab47 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.335672 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeb125-22a9-4271-ade6-70128537ab47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.336656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeb125-22a9-4271-ade6-70128537ab47 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.338362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeeb125-22a9-4271-ade6-70128537ab47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:31.338956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeeb125-22a9-4271-ade6-70128537ab47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:31.342770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeeb125-22a9-4271-ade6-70128537ab47 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:31.379530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1158/2325] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 14:03:31.380290 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2712d7b-15e4-4927-97ac-860f82b1b8b7 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:31.390552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2712d7b-15e4-4927-97ac-860f82b1b8b7 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:31.395711 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42086ca-efaa-45f0-b106-ec8913988d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:31.396375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42086ca-efaa-45f0-b106-ec8913988d02 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.396655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42086ca-efaa-45f0-b106-ec8913988d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.396925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42086ca-efaa-45f0-b106-ec8913988d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.397615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42086ca-efaa-45f0-b106-ec8913988d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:31.399569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42086ca-efaa-45f0-b106-ec8913988d02 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:31.426996 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1168/2326] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:31 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:31.431254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeeb125-22a9-4271-ade6-70128537ab47 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:31.441567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeeb125-22a9-4271-ade6-70128537ab47 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:31.503005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf739e-f811-4938-b3a6-02a8c91001bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:31.503678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf739e-f811-4938-b3a6-02a8c91001bf admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.503957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf739e-f811-4938-b3a6-02a8c91001bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.504229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf739e-f811-4938-b3a6-02a8c91001bf admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.504909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf739e-f811-4938-b3a6-02a8c91001bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:31.506860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf739e-f811-4938-b3a6-02a8c91001bf admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:31.533046 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1159/2327] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:31 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:31.534483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42086ca-efaa-45f0-b106-ec8913988d02 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:31.546016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42086ca-efaa-45f0-b106-ec8913988d02 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:31.552142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c0818-b479-449d-85b4-1c8e1d799d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:31.552853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c0818-b479-449d-85b4-1c8e1d799d97 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.553173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c0818-b479-449d-85b4-1c8e1d799d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.553448 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c0818-b479-449d-85b4-1c8e1d799d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.554169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c0818-b479-449d-85b4-1c8e1d799d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:31.555988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c0818-b479-449d-85b4-1c8e1d799d97 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:31.567872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1169/2328] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:31.571755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf739e-f811-4938-b3a6-02a8c91001bf admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:31.583481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf739e-f811-4938-b3a6-02a8c91001bf admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:31.589303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3a64-af2a-46f3-a588-64f9232d24a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:31.589918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3a64-af2a-46f3-a588-64f9232d24a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.590193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3a64-af2a-46f3-a588-64f9232d24a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.590468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3a64-af2a-46f3-a588-64f9232d24a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.591242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd3a64-af2a-46f3-a588-64f9232d24a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:31.593427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd3a64-af2a-46f3-a588-64f9232d24a0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:31.603929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1160/2329] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:31.608209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c0818-b479-449d-85b4-1c8e1d799d97 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:31.620057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c0818-b479-449d-85b4-1c8e1d799d97 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:31.625644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959ac6b-84c9-4f6e-ad67-2af50a9c1402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:31.626550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959ac6b-84c9-4f6e-ad67-2af50a9c1402 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.626872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959ac6b-84c9-4f6e-ad67-2af50a9c1402 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.627150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959ac6b-84c9-4f6e-ad67-2af50a9c1402 admin admin] PATH_INFO: `/v3/projects/1fcb7d94e2ba453590ff33e773fd990c/users/f1eaf82e62cc445ea8659fd2e8664dbf/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.643018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959ac6b-84c9-4f6e-ad67-2af50a9c1402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fcb7d94e2ba453590ff33e773fd990c, user_id=f1eaf82e62cc445ea8659fd2e8664dbf, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:31.645765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959ac6b-84c9-4f6e-ad67-2af50a9c1402 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:31.660432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1170/2330] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:31 2022] PUT /identity/v3/projects/1fcb7d94e2ba453590ff33e773fd990c/users/f1eaf82e62cc445ea8659fd2e8664dbf/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:31.674115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf56ab-3816-4742-a209-d544608a53cc None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.674639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf56ab-3816-4742-a209-d544608a53cc None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.675074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf56ab-3816-4742-a209-d544608a53cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.708646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f5bf56ab-3816-4742-a209-d544608a53cc None None] MFA Rules not processed for user `f1eaf82e62cc445ea8659fd2e8664dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:31.709882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f5bf56ab-3816-4742-a209-d544608a53cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:31.793197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1161/2331] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:31.798419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959ac6b-84c9-4f6e-ad67-2af50a9c1402 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:31.883286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959ac6b-84c9-4f6e-ad67-2af50a9c1402 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:31.890733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94bf23-fe01-469a-9e9d-dee78fd477c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:31.891611 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94bf23-fe01-469a-9e9d-dee78fd477c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:31.891981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94bf23-fe01-469a-9e9d-dee78fd477c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:31.892548 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94bf23-fe01-469a-9e9d-dee78fd477c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:31.893506 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94bf23-fe01-469a-9e9d-dee78fd477c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:31.895623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94bf23-fe01-469a-9e9d-dee78fd477c9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:31.926347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1171/2332] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:31 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:31.930552 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf56ab-3816-4742-a209-d544608a53cc None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:31.943271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf56ab-3816-4742-a209-d544608a53cc None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:32.006889 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34c897-56ae-4aa9-93f0-2f79a4534bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:32.007577 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34c897-56ae-4aa9-93f0-2f79a4534bfd admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:32.007862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34c897-56ae-4aa9-93f0-2f79a4534bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:32.008137 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34c897-56ae-4aa9-93f0-2f79a4534bfd admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:32.008844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34c897-56ae-4aa9-93f0-2f79a4534bfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:32.010846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34c897-56ae-4aa9-93f0-2f79a4534bfd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:32.032101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1162/2333] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:31 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:32.036878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94bf23-fe01-469a-9e9d-dee78fd477c9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:32.048479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94bf23-fe01-469a-9e9d-dee78fd477c9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:32.053475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f75d36-22a7-4630-a1d2-e983f5b39aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:32.054146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f75d36-22a7-4630-a1d2-e983f5b39aba admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:32.054429 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f75d36-22a7-4630-a1d2-e983f5b39aba admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:32.054800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f75d36-22a7-4630-a1d2-e983f5b39aba admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:32.055465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f75d36-22a7-4630-a1d2-e983f5b39aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:32.057076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f75d36-22a7-4630-a1d2-e983f5b39aba admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:32.068492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1172/2334] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:03:32.072860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34c897-56ae-4aa9-93f0-2f79a4534bfd admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:32.082830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34c897-56ae-4aa9-93f0-2f79a4534bfd admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:32.087907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c34df-b713-4915-829b-6bdb90a83013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:32.088796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c34df-b713-4915-829b-6bdb90a83013 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:32.089074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c34df-b713-4915-829b-6bdb90a83013 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:32.089347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c34df-b713-4915-829b-6bdb90a83013 admin admin] PATH_INFO: `/v3/projects/7951f55a395d4ef1b264dae366b31900/users/d2ed0de1c5b447ac8fc162221101a007/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:32.109695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c34df-b713-4915-829b-6bdb90a83013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7951f55a395d4ef1b264dae366b31900, user_id=d2ed0de1c5b447ac8fc162221101a007, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:32.112866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c34df-b713-4915-829b-6bdb90a83013 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:32.146798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1163/2335] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:32 2022] PUT /identity/v3/projects/7951f55a395d4ef1b264dae366b31900/users/d2ed0de1c5b447ac8fc162221101a007/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:32.151248 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f75d36-22a7-4630-a1d2-e983f5b39aba admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:32.177451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f75d36-22a7-4630-a1d2-e983f5b39aba admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:32.252701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2b225-77cb-4910-97d0-9cc0e9748b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:32.253898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2b225-77cb-4910-97d0-9cc0e9748b50 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:32.254534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2b225-77cb-4910-97d0-9cc0e9748b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:32.255072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2b225-77cb-4910-97d0-9cc0e9748b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:32.256165 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2b225-77cb-4910-97d0-9cc0e9748b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:32.259076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2b225-77cb-4910-97d0-9cc0e9748b50 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:32.273810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1173/2336] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:32.278954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c34df-b713-4915-829b-6bdb90a83013 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:32.293177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c34df-b713-4915-829b-6bdb90a83013 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:32.298753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f08bf-43be-4a53-99d8-2226da7b9554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:32.299800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f08bf-43be-4a53-99d8-2226da7b9554 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:32.300191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f08bf-43be-4a53-99d8-2226da7b9554 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:32.300574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f08bf-43be-4a53-99d8-2226da7b9554 admin admin] PATH_INFO: `/v3/projects/7951f55a395d4ef1b264dae366b31900/users/d2ed0de1c5b447ac8fc162221101a007/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:32.303498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f08bf-43be-4a53-99d8-2226da7b9554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7951f55a395d4ef1b264dae366b31900, user_id=d2ed0de1c5b447ac8fc162221101a007, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:32.306342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f08bf-43be-4a53-99d8-2226da7b9554 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:32.327145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1164/2337] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:32 2022] PUT /identity/v3/projects/7951f55a395d4ef1b264dae366b31900/users/d2ed0de1c5b447ac8fc162221101a007/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:32.332626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6ea7e-bb10-49b5-a939-f37aef3a04af None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:32.332818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6ea7e-bb10-49b5-a939-f37aef3a04af None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:32.333097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6ea7e-bb10-49b5-a939-f37aef3a04af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:32.365996 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-48d6ea7e-bb10-49b5-a939-f37aef3a04af None None] MFA Rules not processed for user `d2ed0de1c5b447ac8fc162221101a007`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:32.367339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-48d6ea7e-bb10-49b5-a939-f37aef3a04af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:32.496210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1174/2338] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:32.498987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f08bf-43be-4a53-99d8-2226da7b9554 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:32.503389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-387f08bf-43be-4a53-99d8-2226da7b9554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:32.513608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6ea7e-bb10-49b5-a939-f37aef3a04af None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:32.513608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-48d6ea7e-bb10-49b5-a939-f37aef3a04af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:32.604721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6ea7e-bb10-49b5-a939-f37aef3a04af None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:32.610362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e753dc2-8725-4e44-91a0-07929c1d1b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:32.611718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753dc2-8725-4e44-91a0-07929c1d1b08 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:32.611718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753dc2-8725-4e44-91a0-07929c1d1b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:32.612034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753dc2-8725-4e44-91a0-07929c1d1b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:32.617034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f08bf-43be-4a53-99d8-2226da7b9554 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:32.624341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:32.625790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:32.626227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:32.626661 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] PATH_INFO: `/v3/users/648ae92126944b50a22ad780e92df914` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:32.628469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=648ae92126944b50a22ad780e92df914)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:32.631935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:32.698550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648ae92126944b50a22ad780e92df914'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:32.699568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648ae92126944b50a22ad780e92df914'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:32.701033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e753dc2-8725-4e44-91a0-07929c1d1b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:32.703631 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e753dc2-8725-4e44-91a0-07929c1d1b08 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:32.716113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648ae92126944b50a22ad780e92df914'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:32.716540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1175/2339] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:32.717334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648ae92126944b50a22ad780e92df914'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:32.718602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e753dc2-8725-4e44-91a0-07929c1d1b08 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:32.720612 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7e753dc2-8725-4e44-91a0-07929c1d1b08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:32.738806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648ae92126944b50a22ad780e92df914'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:32.760870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1165/2340] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:32 2022] DELETE /identity/v3/users/648ae92126944b50a22ad780e92df914 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:32.802169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e753dc2-8725-4e44-91a0-07929c1d1b08 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:32.860350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa442cc-d30f-4f01-90c0-a9a3fdba02c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:32.861009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa442cc-d30f-4f01-90c0-a9a3fdba02c7 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:32.861291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa442cc-d30f-4f01-90c0-a9a3fdba02c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:32.861566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa442cc-d30f-4f01-90c0-a9a3fdba02c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:32.863887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-3fa442cc-d30f-4f01-90c0-a9a3fdba02c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:32.954528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa442cc-d30f-4f01-90c0-a9a3fdba02c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:32.955939 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa442cc-d30f-4f01-90c0-a9a3fdba02c7 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:32.966797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1176/2341] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:32.976941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:32.979981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:33.019522 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3f235-a2aa-4fc8-8fc9-15e78ba12bfd admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:33.025517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:33.026424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.026730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.027046 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] PATH_INFO: `/v3/projects/ac9b3816e0cb4400919797c107065ad4` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.028282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9b3816e0cb4400919797c107065ad4)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:33.030926 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:33.088094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9b3816e0cb4400919797c107065ad4', 'request_id': 'req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:33.096395 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9b3816e0cb4400919797c107065ad4', 'request_id': 'req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:33.105221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9b3816e0cb4400919797c107065ad4', 'request_id': 'req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:33.128407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] The token cache is being invalidate because project ac9b3816e0cb4400919797c107065ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:33.129345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:33.129943 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b6b594b8-0041-4332-bb7b-3b3c01c1fa7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:33.137116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1166/2342] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:32 2022] DELETE /identity/v3/projects/ac9b3816e0cb4400919797c107065ad4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:33.140552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f04ea-3704-4936-85db-8aaa16b39c95 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.141100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f04ea-3704-4936-85db-8aaa16b39c95 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.141859 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f04ea-3704-4936-85db-8aaa16b39c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.174093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-8c1f04ea-3704-4936-85db-8aaa16b39c95 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:33.176362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-8c1f04ea-3704-4936-85db-8aaa16b39c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:33.290160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1177/2343] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:33.298399 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12bb83-82ea-434e-a37f-1a73997cfb2f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.298802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12bb83-82ea-434e-a37f-1a73997cfb2f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.299166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12bb83-82ea-434e-a37f-1a73997cfb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.319495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-fc12bb83-82ea-434e-a37f-1a73997cfb2f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:33.320858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-fc12bb83-82ea-434e-a37f-1a73997cfb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:33.329789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1167/2344] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:33.334127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f04ea-3704-4936-85db-8aaa16b39c95 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:33.344608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f04ea-3704-4936-85db-8aaa16b39c95 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:33.349648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6f12a-5f64-4cf6-ac03-2a617614578c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:33.350322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6f12a-5f64-4cf6-ac03-2a617614578c admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.350667 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6f12a-5f64-4cf6-ac03-2a617614578c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.350980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6f12a-5f64-4cf6-ac03-2a617614578c admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.351713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6f12a-5f64-4cf6-ac03-2a617614578c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:33.352024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6f12a-5f64-4cf6-ac03-2a617614578c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:33.353818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6f12a-5f64-4cf6-ac03-2a617614578c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:33.368778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1178/2345] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:03:33.373217 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12bb83-82ea-434e-a37f-1a73997cfb2f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:33.382379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12bb83-82ea-434e-a37f-1a73997cfb2f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:33.387642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a312ba-0bed-4194-bf38-d05f0ff24694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:33.388298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a312ba-0bed-4194-bf38-d05f0ff24694 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.388574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a312ba-0bed-4194-bf38-d05f0ff24694 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.388951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a312ba-0bed-4194-bf38-d05f0ff24694 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.389545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a312ba-0bed-4194-bf38-d05f0ff24694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:33.392035 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a312ba-0bed-4194-bf38-d05f0ff24694 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:33.419535 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1168/2346] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:33 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:33.423858 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6f12a-5f64-4cf6-ac03-2a617614578c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:33.437477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6f12a-5f64-4cf6-ac03-2a617614578c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:33.502170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4b045-cbb1-403b-ae75-5c912ebf89d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:33.502798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4b045-cbb1-403b-ae75-5c912ebf89d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.503081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4b045-cbb1-403b-ae75-5c912ebf89d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.503351 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4b045-cbb1-403b-ae75-5c912ebf89d1 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.504069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4b045-cbb1-403b-ae75-5c912ebf89d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:33.506114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4b045-cbb1-403b-ae75-5c912ebf89d1 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:33.534252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1179/2347] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:33 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:33.535696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a312ba-0bed-4194-bf38-d05f0ff24694 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:33.551538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a312ba-0bed-4194-bf38-d05f0ff24694 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:33.556920 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221105d7-9896-4f92-9b9b-5201533af3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:33.557598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221105d7-9896-4f92-9b9b-5201533af3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.557899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221105d7-9896-4f92-9b9b-5201533af3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.558181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221105d7-9896-4f92-9b9b-5201533af3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.558978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221105d7-9896-4f92-9b9b-5201533af3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:33.560557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221105d7-9896-4f92-9b9b-5201533af3a1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:33.572239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1169/2348] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:33.576286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4b045-cbb1-403b-ae75-5c912ebf89d1 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:33.586887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4b045-cbb1-403b-ae75-5c912ebf89d1 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:33.592074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f8ae7-e684-4c3a-a61d-8f86bbf086c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:33.592742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f8ae7-e684-4c3a-a61d-8f86bbf086c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.593662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f8ae7-e684-4c3a-a61d-8f86bbf086c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.593943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f8ae7-e684-4c3a-a61d-8f86bbf086c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.594671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f8ae7-e684-4c3a-a61d-8f86bbf086c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:33.596299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f8ae7-e684-4c3a-a61d-8f86bbf086c6 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:33.607102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1180/2349] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:33.611459 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221105d7-9896-4f92-9b9b-5201533af3a1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:33.622594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221105d7-9896-4f92-9b9b-5201533af3a1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:33.628107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271ad4e-ffb0-485d-b604-b7782d0e01af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:33.629031 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271ad4e-ffb0-485d-b604-b7782d0e01af admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.629315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271ad4e-ffb0-485d-b604-b7782d0e01af admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.629567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271ad4e-ffb0-485d-b604-b7782d0e01af admin admin] PATH_INFO: `/v3/projects/63364337ddef4aa89de6901cefef1758/users/481b3ea0181e4b5487724aa3b640be84/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.645156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271ad4e-ffb0-485d-b604-b7782d0e01af admin admin] RBAC: Authorizing `identity:create_grant(project_id=63364337ddef4aa89de6901cefef1758, user_id=481b3ea0181e4b5487724aa3b640be84, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:33.648277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271ad4e-ffb0-485d-b604-b7782d0e01af admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:33.661867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1170/2350] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:33 2022] PUT /identity/v3/projects/63364337ddef4aa89de6901cefef1758/users/481b3ea0181e4b5487724aa3b640be84/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:33.673087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e385f5-6d80-4216-95ff-501c7ad43a33 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.673291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e385f5-6d80-4216-95ff-501c7ad43a33 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.673424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e385f5-6d80-4216-95ff-501c7ad43a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.700766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a4e385f5-6d80-4216-95ff-501c7ad43a33 None None] MFA Rules not processed for user `481b3ea0181e4b5487724aa3b640be84`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:33.701982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a4e385f5-6d80-4216-95ff-501c7ad43a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:33.767516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1181/2351] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:33.771785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271ad4e-ffb0-485d-b604-b7782d0e01af admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:33.865275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271ad4e-ffb0-485d-b604-b7782d0e01af admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:33.870192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619cc3b-7cb5-427b-8bed-85367b53bfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:33.870953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619cc3b-7cb5-427b-8bed-85367b53bfe1 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.871238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619cc3b-7cb5-427b-8bed-85367b53bfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.871512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619cc3b-7cb5-427b-8bed-85367b53bfe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.872220 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619cc3b-7cb5-427b-8bed-85367b53bfe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:33.874865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619cc3b-7cb5-427b-8bed-85367b53bfe1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:33.902110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1171/2352] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:33 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:33.906269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e385f5-6d80-4216-95ff-501c7ad43a33 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:33.917649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e385f5-6d80-4216-95ff-501c7ad43a33 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:33.984514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b102a24-593c-4e4f-a72f-965e2eeb80cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:33.985196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b102a24-593c-4e4f-a72f-965e2eeb80cd admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:33.985472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b102a24-593c-4e4f-a72f-965e2eeb80cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:33.985738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b102a24-593c-4e4f-a72f-965e2eeb80cd admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:33.986435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b102a24-593c-4e4f-a72f-965e2eeb80cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:33.988393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b102a24-593c-4e4f-a72f-965e2eeb80cd admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:34.013081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1182/2353] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:33 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:34.015050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619cc3b-7cb5-427b-8bed-85367b53bfe1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:34.025826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619cc3b-7cb5-427b-8bed-85367b53bfe1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:34.031066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca86aa-b71f-4394-98d7-543d08bdd063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:34.031749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca86aa-b71f-4394-98d7-543d08bdd063 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:34.032024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca86aa-b71f-4394-98d7-543d08bdd063 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:34.032293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca86aa-b71f-4394-98d7-543d08bdd063 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:34.033015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca86aa-b71f-4394-98d7-543d08bdd063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:34.034736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca86aa-b71f-4394-98d7-543d08bdd063 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:34.048466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1172/2354] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:03:34.052623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b102a24-593c-4e4f-a72f-965e2eeb80cd admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:34.064247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b102a24-593c-4e4f-a72f-965e2eeb80cd admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:34.069600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef1015-4a19-42cb-b472-61fb8d24fe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:34.070515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef1015-4a19-42cb-b472-61fb8d24fe28 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:34.071083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef1015-4a19-42cb-b472-61fb8d24fe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:34.071356 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef1015-4a19-42cb-b472-61fb8d24fe28 admin admin] PATH_INFO: `/v3/projects/a0983f6d364a471a9e4ada349eaacb14/users/32c307e61f364fab9d1bd82960bebc33/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:34.087846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef1015-4a19-42cb-b472-61fb8d24fe28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0983f6d364a471a9e4ada349eaacb14, user_id=32c307e61f364fab9d1bd82960bebc33, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:34.090886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef1015-4a19-42cb-b472-61fb8d24fe28 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:34.109988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1183/2355] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:34 2022] PUT /identity/v3/projects/a0983f6d364a471a9e4ada349eaacb14/users/32c307e61f364fab9d1bd82960bebc33/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:34.111305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca86aa-b71f-4394-98d7-543d08bdd063 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:34.123559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca86aa-b71f-4394-98d7-543d08bdd063 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:34.201625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01110bd-2fff-475d-acc3-49706c5bb136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:34.203003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01110bd-2fff-475d-acc3-49706c5bb136 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:34.203287 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01110bd-2fff-475d-acc3-49706c5bb136 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:34.203565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01110bd-2fff-475d-acc3-49706c5bb136 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:34.204290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01110bd-2fff-475d-acc3-49706c5bb136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:34.205978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01110bd-2fff-475d-acc3-49706c5bb136 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:34.216374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1173/2356] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:34.220878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef1015-4a19-42cb-b472-61fb8d24fe28 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:34.231767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef1015-4a19-42cb-b472-61fb8d24fe28 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:34.238200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df3d0a-de34-4dfb-ab54-d56ad10bd9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:34.239130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df3d0a-de34-4dfb-ab54-d56ad10bd9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:34.239412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df3d0a-de34-4dfb-ab54-d56ad10bd9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:34.239686 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df3d0a-de34-4dfb-ab54-d56ad10bd9e6 admin admin] PATH_INFO: `/v3/projects/a0983f6d364a471a9e4ada349eaacb14/users/32c307e61f364fab9d1bd82960bebc33/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:34.241752 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df3d0a-de34-4dfb-ab54-d56ad10bd9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0983f6d364a471a9e4ada349eaacb14, user_id=32c307e61f364fab9d1bd82960bebc33, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:34.244592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df3d0a-de34-4dfb-ab54-d56ad10bd9e6 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:34.255635 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1184/2357] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:34 2022] PUT /identity/v3/projects/a0983f6d364a471a9e4ada349eaacb14/users/32c307e61f364fab9d1bd82960bebc33/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:34.268525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713ddf0-bf7b-463e-b7b8-daaf7e9b126c None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:34.268928 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713ddf0-bf7b-463e-b7b8-daaf7e9b126c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:34.269320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713ddf0-bf7b-463e-b7b8-daaf7e9b126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:34.310501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1713ddf0-bf7b-463e-b7b8-daaf7e9b126c None None] MFA Rules not processed for user `32c307e61f364fab9d1bd82960bebc33`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:34.312124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1713ddf0-bf7b-463e-b7b8-daaf7e9b126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:34.405685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1174/2358] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:34.417842 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df3d0a-de34-4dfb-ab54-d56ad10bd9e6 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:34.419899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-27df3d0a-de34-4dfb-ab54-d56ad10bd9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:34.495083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df3d0a-de34-4dfb-ab54-d56ad10bd9e6 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:34.499202 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bac17-dac0-48e4-b182-d48ceaa199ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:34.500039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bac17-dac0-48e4-b182-d48ceaa199ce service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:34.500336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bac17-dac0-48e4-b182-d48ceaa199ce service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:34.500628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bac17-dac0-48e4-b182-d48ceaa199ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:34.511380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bac17-dac0-48e4-b182-d48ceaa199ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:34.512718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bac17-dac0-48e4-b182-d48ceaa199ce service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:34.524663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1185/2359] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:34.562209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713ddf0-bf7b-463e-b7b8-daaf7e9b126c None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:34.666901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713ddf0-bf7b-463e-b7b8-daaf7e9b126c None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:34.674396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:34.675366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:34.675649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:34.675925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] PATH_INFO: `/v3/users/481b3ea0181e4b5487724aa3b640be84` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:34.677879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=481b3ea0181e4b5487724aa3b640be84)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:34.680219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:34.747988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481b3ea0181e4b5487724aa3b640be84'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:34.748776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481b3ea0181e4b5487724aa3b640be84'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:34.763119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481b3ea0181e4b5487724aa3b640be84'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:34.763891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481b3ea0181e4b5487724aa3b640be84'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:34.772772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481b3ea0181e4b5487724aa3b640be84'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:34.789264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1175/2360] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:34 2022] DELETE /identity/v3/users/481b3ea0181e4b5487724aa3b640be84 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:34.793819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859e1d0-d0b7-4e9a-81f0-a48b786776f1 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:34.794106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859e1d0-d0b7-4e9a-81f0-a48b786776f1 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:34.794385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859e1d0-d0b7-4e9a-81f0-a48b786776f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:34.816416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2859e1d0-d0b7-4e9a-81f0-a48b786776f1 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:34.817886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2859e1d0-d0b7-4e9a-81f0-a48b786776f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:34.899796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1186/2361] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:34.915749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:34.917338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:35.007791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509768af-6526-4575-8f12-5060a2d78d26 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:35.014083 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719e5c0-6b82-4ae5-96ee-1c5212119fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:35.015221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719e5c0-6b82-4ae5-96ee-1c5212119fcb service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:35.015221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719e5c0-6b82-4ae5-96ee-1c5212119fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:35.015427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719e5c0-6b82-4ae5-96ee-1c5212119fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:35.029107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719e5c0-6b82-4ae5-96ee-1c5212119fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:35.030763 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719e5c0-6b82-4ae5-96ee-1c5212119fcb service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:35.041381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1176/2362] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:35.521283 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859e1d0-d0b7-4e9a-81f0-a48b786776f1 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:35.522957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2859e1d0-d0b7-4e9a-81f0-a48b786776f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:35.536389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859e1d0-d0b7-4e9a-81f0-a48b786776f1 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:35.542419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:35.543529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:35.543932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:35.544306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] PATH_INFO: `/v3/projects/63364337ddef4aa89de6901cefef1758` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:35.546617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=63364337ddef4aa89de6901cefef1758)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:35.548541 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:35.614825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63364337ddef4aa89de6901cefef1758', 'request_id': 'req-bf04c942-7983-4bf9-88f9-669fc32daa1a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:35.621870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63364337ddef4aa89de6901cefef1758', 'request_id': 'req-bf04c942-7983-4bf9-88f9-669fc32daa1a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:35.630607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63364337ddef4aa89de6901cefef1758', 'request_id': 'req-bf04c942-7983-4bf9-88f9-669fc32daa1a'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:35.645048 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] The token cache is being invalidate because project 63364337ddef4aa89de6901cefef1758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:35.645340 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:35.646021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:35.647701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1187/2363] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:35 2022] DELETE /identity/v3/projects/63364337ddef4aa89de6901cefef1758 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:35.651823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719e5c0-6b82-4ae5-96ee-1c5212119fcb service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:35.664830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719e5c0-6b82-4ae5-96ee-1c5212119fcb service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:35.731504 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:35.732476 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:35.732934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:35.733108 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] PATH_INFO: `/v3/users/32c307e61f364fab9d1bd82960bebc33` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:35.734629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c307e61f364fab9d1bd82960bebc33)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:35.738099 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:35.816955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c307e61f364fab9d1bd82960bebc33'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:35.817989 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c307e61f364fab9d1bd82960bebc33'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:35.842223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c307e61f364fab9d1bd82960bebc33'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:35.843451 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c307e61f364fab9d1bd82960bebc33'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:35.854557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1ece0301-8a0d-4857-8165-0c543de44051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c307e61f364fab9d1bd82960bebc33'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:35.877967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1177/2364] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:35 2022] DELETE /identity/v3/users/32c307e61f364fab9d1bd82960bebc33 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:36.484882 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:36.486388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:36.502593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04c942-7983-4bf9-88f9-669fc32daa1a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:36.570240 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:36.571041 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:36.571321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:36.571723 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] PATH_INFO: `/v3/projects/a0983f6d364a471a9e4ada349eaacb14` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:36.572789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0983f6d364a471a9e4ada349eaacb14)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:36.576357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:36.632033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0983f6d364a471a9e4ada349eaacb14', 'request_id': 'req-66ea7a46-8bd1-453d-ba21-aede573e1497'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:36.638368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0983f6d364a471a9e4ada349eaacb14', 'request_id': 'req-66ea7a46-8bd1-453d-ba21-aede573e1497'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:36.648236 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0983f6d364a471a9e4ada349eaacb14', 'request_id': 'req-66ea7a46-8bd1-453d-ba21-aede573e1497'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:36.662266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] The token cache is being invalidate because project a0983f6d364a471a9e4ada349eaacb14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:36.662562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:36.663349 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-66ea7a46-8bd1-453d-ba21-aede573e1497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:36.668535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1188/2365] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:36 2022] DELETE /identity/v3/projects/a0983f6d364a471a9e4ada349eaacb14 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:36.671892 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4312b0b-cbad-4c91-905b-b5fef935bbf5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:36.672437 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4312b0b-cbad-4c91-905b-b5fef935bbf5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:36.672557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4312b0b-cbad-4c91-905b-b5fef935bbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:36.712876 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f4312b0b-cbad-4c91-905b-b5fef935bbf5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:36.714229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f4312b0b-cbad-4c91-905b-b5fef935bbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:36.798053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1178/2366] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:36.805742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7889ff-df17-49be-85e9-dc981a6e75be None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:36.806112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7889ff-df17-49be-85e9-dc981a6e75be None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:36.806363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7889ff-df17-49be-85e9-dc981a6e75be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:36.826045 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-9a7889ff-df17-49be-85e9-dc981a6e75be None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:36.827046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9a7889ff-df17-49be-85e9-dc981a6e75be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:36.838850 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1189/2367] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:36.843312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4312b0b-cbad-4c91-905b-b5fef935bbf5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:36.853232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4312b0b-cbad-4c91-905b-b5fef935bbf5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:36.858250 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaf5d0-fed8-4119-84f2-dbce62ece7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:36.859016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaf5d0-fed8-4119-84f2-dbce62ece7ce admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:36.859300 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaf5d0-fed8-4119-84f2-dbce62ece7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:36.859574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaf5d0-fed8-4119-84f2-dbce62ece7ce admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:36.860334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdaf5d0-fed8-4119-84f2-dbce62ece7ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:36.860637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdaf5d0-fed8-4119-84f2-dbce62ece7ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:36.862246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdaf5d0-fed8-4119-84f2-dbce62ece7ce admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:36.874935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1179/2368] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:03:36.879287 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7889ff-df17-49be-85e9-dc981a6e75be None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:36.888775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7889ff-df17-49be-85e9-dc981a6e75be None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:36.894688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90717b0b-7066-4091-98a8-f3d83cf3bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:36.895230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717b0b-7066-4091-98a8-f3d83cf3bc51 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:36.895510 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717b0b-7066-4091-98a8-f3d83cf3bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:36.895787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717b0b-7066-4091-98a8-f3d83cf3bc51 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:36.896479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717b0b-7066-4091-98a8-f3d83cf3bc51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:36.898408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717b0b-7066-4091-98a8-f3d83cf3bc51 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:36.902199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaf5d0-fed8-4119-84f2-dbce62ece7ce admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:36.904346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7bdaf5d0-fed8-4119-84f2-dbce62ece7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:36.918596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaf5d0-fed8-4119-84f2-dbce62ece7ce admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:36.924324 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:36.925192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:36.925469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:36.925744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] PATH_INFO: `/v3/users/a37f5d5ddd9845fbaf7d5645c31b2d40` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:36.927364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a37f5d5ddd9845fbaf7d5645c31b2d40)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:36.929305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:36.932614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1190/2369] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:36 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:36.933655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90717b0b-7066-4091-98a8-f3d83cf3bc51 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:36.979235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a37f5d5ddd9845fbaf7d5645c31b2d40'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:36.979923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a37f5d5ddd9845fbaf7d5645c31b2d40'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:36.993086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a37f5d5ddd9845fbaf7d5645c31b2d40'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:36.993910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a37f5d5ddd9845fbaf7d5645c31b2d40'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:36.997873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90717b0b-7066-4091-98a8-f3d83cf3bc51 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.001540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a37f5d5ddd9845fbaf7d5645c31b2d40'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:37.003184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc009564-dd54-4cd1-a912-b3a1bb07baca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.003844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc009564-dd54-4cd1-a912-b3a1bb07baca admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.004131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc009564-dd54-4cd1-a912-b3a1bb07baca admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.004403 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc009564-dd54-4cd1-a912-b3a1bb07baca admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.005084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc009564-dd54-4cd1-a912-b3a1bb07baca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.007214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc009564-dd54-4cd1-a912-b3a1bb07baca admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.015252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1180/2370] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:36 2022] DELETE /identity/v3/users/a37f5d5ddd9845fbaf7d5645c31b2d40 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:37.030744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1191/2371] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:36 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:37.033357 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.034899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:37.099291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96334a63-402c-4b63-ba5e-e3c2848865cd admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.103805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f42e0e-3dc5-4aa0-b5db-b068b9709e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.104466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f42e0e-3dc5-4aa0-b5db-b068b9709e51 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.104738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f42e0e-3dc5-4aa0-b5db-b068b9709e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.105007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f42e0e-3dc5-4aa0-b5db-b068b9709e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.105714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f42e0e-3dc5-4aa0-b5db-b068b9709e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.107348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f42e0e-3dc5-4aa0-b5db-b068b9709e51 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.118058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1181/2372] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:37.122154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc009564-dd54-4cd1-a912-b3a1bb07baca admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.132645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc009564-dd54-4cd1-a912-b3a1bb07baca admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.137448 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa459f-6833-4ea5-9a15-576f717519ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.138123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa459f-6833-4ea5-9a15-576f717519ed admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.138589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa459f-6833-4ea5-9a15-576f717519ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.138909 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa459f-6833-4ea5-9a15-576f717519ed admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.139676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa459f-6833-4ea5-9a15-576f717519ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.141821 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa459f-6833-4ea5-9a15-576f717519ed admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.152281 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1192/2373] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:37.156834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f42e0e-3dc5-4aa0-b5db-b068b9709e51 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.167023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f42e0e-3dc5-4aa0-b5db-b068b9709e51 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.172990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5b02d-ae4c-45cd-8a13-3db7e4cbdc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.173417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5b02d-ae4c-45cd-8a13-3db7e4cbdc9f admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.173654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5b02d-ae4c-45cd-8a13-3db7e4cbdc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.174031 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5b02d-ae4c-45cd-8a13-3db7e4cbdc9f admin admin] PATH_INFO: `/v3/projects/fc755d186d004ab3805c54d8f41cc982/users/c1b4f67445db47c9bcdeb6f5df6e9705/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.190901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5b02d-ae4c-45cd-8a13-3db7e4cbdc9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc755d186d004ab3805c54d8f41cc982, user_id=c1b4f67445db47c9bcdeb6f5df6e9705, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.194329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5b02d-ae4c-45cd-8a13-3db7e4cbdc9f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.195120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa459f-6833-4ea5-9a15-576f717519ed admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.197299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e7aa459f-6833-4ea5-9a15-576f717519ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:37.212227 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1182/2374] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:37 2022] PUT /identity/v3/projects/fc755d186d004ab3805c54d8f41cc982/users/c1b4f67445db47c9bcdeb6f5df6e9705/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:37.228112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19808ba2-efe4-4506-a37a-c59407af25da None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.228112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19808ba2-efe4-4506-a37a-c59407af25da None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.228112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19808ba2-efe4-4506-a37a-c59407af25da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.228812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa459f-6833-4ea5-9a15-576f717519ed admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.258728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-19808ba2-efe4-4506-a37a-c59407af25da None None] MFA Rules not processed for user `c1b4f67445db47c9bcdeb6f5df6e9705`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:37.259973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-19808ba2-efe4-4506-a37a-c59407af25da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:37.305684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.306550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.306880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.307220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] PATH_INFO: `/v3/projects/ef87180052d643af940a8e2b4f21e46f` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.308425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef87180052d643af940a8e2b4f21e46f)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.310324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.335902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1183/2375] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:37.336942 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19808ba2-efe4-4506-a37a-c59407af25da None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.338820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-19808ba2-efe4-4506-a37a-c59407af25da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:37.357700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef87180052d643af940a8e2b4f21e46f', 'request_id': 'req-dedef271-7d3b-4e1b-851f-1217d2316c04'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:37.364286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef87180052d643af940a8e2b4f21e46f', 'request_id': 'req-dedef271-7d3b-4e1b-851f-1217d2316c04'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:37.373544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef87180052d643af940a8e2b4f21e46f', 'request_id': 'req-dedef271-7d3b-4e1b-851f-1217d2316c04'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:37.386555 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] The token cache is being invalidate because project ef87180052d643af940a8e2b4f21e46f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:37.387002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:37.387935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:37.389698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1193/2376] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:37 2022] DELETE /identity/v3/projects/ef87180052d643af940a8e2b4f21e46f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:37.391454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.392809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:37.397197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19808ba2-efe4-4506-a37a-c59407af25da None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.451345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8786c2-8198-4a53-b419-e87f34e65ff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.452022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8786c2-8198-4a53-b419-e87f34e65ff4 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.452301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8786c2-8198-4a53-b419-e87f34e65ff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.452583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8786c2-8198-4a53-b419-e87f34e65ff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.454439 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-fc8786c2-8198-4a53-b419-e87f34e65ff4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:37.464241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedef271-7d3b-4e1b-851f-1217d2316c04 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.470610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d70059-8569-42b8-98c1-ebb78bacbaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.471322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d70059-8569-42b8-98c1-ebb78bacbaa4 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.471608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d70059-8569-42b8-98c1-ebb78bacbaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.471883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d70059-8569-42b8-98c1-ebb78bacbaa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.473546 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d70059-8569-42b8-98c1-ebb78bacbaa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.474539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d70059-8569-42b8-98c1-ebb78bacbaa4 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.502755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1194/2377] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:37 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:37.503313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5eb10-f78f-4a83-8828-d4834bd6c6f1 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.503602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5eb10-f78f-4a83-8828-d4834bd6c6f1 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.504020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5eb10-f78f-4a83-8828-d4834bd6c6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.520364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8786c2-8198-4a53-b419-e87f34e65ff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.522928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e8e5eb10-f78f-4a83-8828-d4834bd6c6f1 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:37.523105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8786c2-8198-4a53-b419-e87f34e65ff4 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.524157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e8e5eb10-f78f-4a83-8828-d4834bd6c6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:37.537671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1184/2378] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:37.539759 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8786c2-8198-4a53-b419-e87f34e65ff4 service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.551516 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8786c2-8198-4a53-b419-e87f34e65ff4 service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.591699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1195/2379] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:37.600335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15104b05-a040-4f27-a12b-ab6695d5fd07 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.600610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15104b05-a040-4f27-a12b-ab6695d5fd07 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.600896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15104b05-a040-4f27-a12b-ab6695d5fd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.610181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec6133-42ba-4e75-b3f5-8e05feb3f20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.610811 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec6133-42ba-4e75-b3f5-8e05feb3f20f admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.611092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec6133-42ba-4e75-b3f5-8e05feb3f20f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.611365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec6133-42ba-4e75-b3f5-8e05feb3f20f admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.612264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec6133-42ba-4e75-b3f5-8e05feb3f20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.613955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec6133-42ba-4e75-b3f5-8e05feb3f20f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.622368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-15104b05-a040-4f27-a12b-ab6695d5fd07 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:37.622368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-15104b05-a040-4f27-a12b-ab6695d5fd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:37.632504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1196/2380] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:37.636832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15104b05-a040-4f27-a12b-ab6695d5fd07 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.640657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1185/2381] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:37 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:37.641579 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec6133-42ba-4e75-b3f5-8e05feb3f20f admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.646399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15104b05-a040-4f27-a12b-ab6695d5fd07 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.651763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d71847-e76c-4360-9f78-a2f0c830212f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.652144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec6133-42ba-4e75-b3f5-8e05feb3f20f admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.652422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d71847-e76c-4360-9f78-a2f0c830212f admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.652699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d71847-e76c-4360-9f78-a2f0c830212f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.652968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d71847-e76c-4360-9f78-a2f0c830212f admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.653860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d71847-e76c-4360-9f78-a2f0c830212f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:37.654161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d71847-e76c-4360-9f78-a2f0c830212f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.656385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d71847-e76c-4360-9f78-a2f0c830212f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.658624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608df88-94dc-4791-97c3-52d0637c7c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.660089 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608df88-94dc-4791-97c3-52d0637c7c62 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.660401 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608df88-94dc-4791-97c3-52d0637c7c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.660677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608df88-94dc-4791-97c3-52d0637c7c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.661402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608df88-94dc-4791-97c3-52d0637c7c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.663043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608df88-94dc-4791-97c3-52d0637c7c62 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.674013 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1197/2382] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 14:03:37.675130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d71847-e76c-4360-9f78-a2f0c830212f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.679188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1186/2383] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:37.683844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608df88-94dc-4791-97c3-52d0637c7c62 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.689600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d71847-e76c-4360-9f78-a2f0c830212f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.694529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608df88-94dc-4791-97c3-52d0637c7c62 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.696704 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b9b0-e7e5-40c4-9eb2-ca9b00be5944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.697405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b9b0-e7e5-40c4-9eb2-ca9b00be5944 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.697727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b9b0-e7e5-40c4-9eb2-ca9b00be5944 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.699702 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b9b0-e7e5-40c4-9eb2-ca9b00be5944 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.701096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c2861-3a12-4c86-9cf6-b6d79c953c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.702575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749b9b0-e7e5-40c4-9eb2-ca9b00be5944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.702859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c2861-3a12-4c86-9cf6-b6d79c953c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.703140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c2861-3a12-4c86-9cf6-b6d79c953c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.703500 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c2861-3a12-4c86-9cf6-b6d79c953c10 admin admin] PATH_INFO: `/v3/projects/058c95cb7bae4085962b74fcbb78ad63/users/1edb2918ae3b4256bb96a5067bfccdb3/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.704778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749b9b0-e7e5-40c4-9eb2-ca9b00be5944 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.721562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c2861-3a12-4c86-9cf6-b6d79c953c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058c95cb7bae4085962b74fcbb78ad63, user_id=1edb2918ae3b4256bb96a5067bfccdb3, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.725173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c2861-3a12-4c86-9cf6-b6d79c953c10 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.735755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1198/2384] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:37 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:37.739703 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b9b0-e7e5-40c4-9eb2-ca9b00be5944 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.741284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1187/2385] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:37 2022] PUT /identity/v3/projects/058c95cb7bae4085962b74fcbb78ad63/users/1edb2918ae3b4256bb96a5067bfccdb3/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:37.745400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c2861-3a12-4c86-9cf6-b6d79c953c10 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.830689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b9b0-e7e5-40c4-9eb2-ca9b00be5944 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.834839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584e337-6064-4311-a322-849a1a43b594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.835893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584e337-6064-4311-a322-849a1a43b594 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.836267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584e337-6064-4311-a322-849a1a43b594 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.837180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584e337-6064-4311-a322-849a1a43b594 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.837987 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584e337-6064-4311-a322-849a1a43b594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.838446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c2861-3a12-4c86-9cf6-b6d79c953c10 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.840977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584e337-6064-4311-a322-849a1a43b594 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.845733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6073d6b-fa03-46e3-9efe-6f20c02d9d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.846536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073d6b-fa03-46e3-9efe-6f20c02d9d2a admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.846847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073d6b-fa03-46e3-9efe-6f20c02d9d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.847180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073d6b-fa03-46e3-9efe-6f20c02d9d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.848111 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6073d6b-fa03-46e3-9efe-6f20c02d9d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.850058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6073d6b-fa03-46e3-9efe-6f20c02d9d2a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.868937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1188/2386] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:37.868937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6073d6b-fa03-46e3-9efe-6f20c02d9d2a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.870314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1199/2387] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:37 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:37.871105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584e337-6064-4311-a322-849a1a43b594 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.880621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6073d6b-fa03-46e3-9efe-6f20c02d9d2a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.881447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584e337-6064-4311-a322-849a1a43b594 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.886426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609f4cb-bd5c-4ffa-b8e8-44d59733e6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.887124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609f4cb-bd5c-4ffa-b8e8-44d59733e6cc admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.887382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e18fd1-afb7-46ba-a396-26e6cd4218ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.887514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609f4cb-bd5c-4ffa-b8e8-44d59733e6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.887684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609f4cb-bd5c-4ffa-b8e8-44d59733e6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.888405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609f4cb-bd5c-4ffa-b8e8-44d59733e6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.889262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e18fd1-afb7-46ba-a396-26e6cd4218ca admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.889689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e18fd1-afb7-46ba-a396-26e6cd4218ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.890576 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609f4cb-bd5c-4ffa-b8e8-44d59733e6cc admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.890818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e18fd1-afb7-46ba-a396-26e6cd4218ca admin admin] PATH_INFO: `/v3/projects/058c95cb7bae4085962b74fcbb78ad63/users/1edb2918ae3b4256bb96a5067bfccdb3/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.894315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e18fd1-afb7-46ba-a396-26e6cd4218ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=058c95cb7bae4085962b74fcbb78ad63, user_id=1edb2918ae3b4256bb96a5067bfccdb3, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:37.897581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e18fd1-afb7-46ba-a396-26e6cd4218ca admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:37.902190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1200/2388] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:37.906515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609f4cb-bd5c-4ffa-b8e8-44d59733e6cc admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:37.916987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1189/2389] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:37 2022] PUT /identity/v3/projects/058c95cb7bae4085962b74fcbb78ad63/users/1edb2918ae3b4256bb96a5067bfccdb3/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:37.918984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609f4cb-bd5c-4ffa-b8e8-44d59733e6cc admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:37.929454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c708c-c294-43d3-a108-222413cdacf6 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.929788 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c708c-c294-43d3-a108-222413cdacf6 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.930005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c708c-c294-43d3-a108-222413cdacf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:37.965077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-847c708c-c294-43d3-a108-222413cdacf6 None None] MFA Rules not processed for user `1edb2918ae3b4256bb96a5067bfccdb3`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:37.966878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-847c708c-c294-43d3-a108-222413cdacf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:37.996750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f59c7c-9c76-4f34-a876-794b6b4d96dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:37.997846 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f59c7c-9c76-4f34-a876-794b6b4d96dc admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:37.998255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f59c7c-9c76-4f34-a876-794b6b4d96dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:37.998635 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f59c7c-9c76-4f34-a876-794b6b4d96dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:38.000125 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f59c7c-9c76-4f34-a876-794b6b4d96dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:38.001839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f59c7c-9c76-4f34-a876-794b6b4d96dc admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:38.012296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1201/2390] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:03:38.016472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f59c7c-9c76-4f34-a876-794b6b4d96dc admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:38.026747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f59c7c-9c76-4f34-a876-794b6b4d96dc admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:38.031391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb23c5d-3daa-4244-ae2f-027ce408c04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:38.032337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb23c5d-3daa-4244-ae2f-027ce408c04d admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:38.032620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb23c5d-3daa-4244-ae2f-027ce408c04d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:38.032895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb23c5d-3daa-4244-ae2f-027ce408c04d admin admin] PATH_INFO: `/v3/projects/9eb67099abec43c59e2aa5fe2f0920ab/users/10d7a3891aa244508dd55484ebb235cb/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:38.050200 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1190/2391] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:38.058233 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb23c5d-3daa-4244-ae2f-027ce408c04d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb67099abec43c59e2aa5fe2f0920ab, user_id=10d7a3891aa244508dd55484ebb235cb, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:38.061083 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb23c5d-3daa-4244-ae2f-027ce408c04d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:38.063264 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c708c-c294-43d3-a108-222413cdacf6 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:38.065376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-847c708c-c294-43d3-a108-222413cdacf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:38.078459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1202/2392] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:38 2022] PUT /identity/v3/projects/9eb67099abec43c59e2aa5fe2f0920ab/users/10d7a3891aa244508dd55484ebb235cb/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:38.089027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3815ffc-f07a-46b5-a49f-c6c49041f4ab None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:38.089297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3815ffc-f07a-46b5-a49f-c6c49041f4ab None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:38.089574 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3815ffc-f07a-46b5-a49f-c6c49041f4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:38.117998 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a3815ffc-f07a-46b5-a49f-c6c49041f4ab None None] MFA Rules not processed for user `10d7a3891aa244508dd55484ebb235cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:38.119338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a3815ffc-f07a-46b5-a49f-c6c49041f4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:38.134152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c708c-c294-43d3-a108-222413cdacf6 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:38.189129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86bcb8-1b35-4a10-9987-3540f1b20af2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:38.190671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86bcb8-1b35-4a10-9987-3540f1b20af2 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:38.191198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86bcb8-1b35-4a10-9987-3540f1b20af2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:38.191590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86bcb8-1b35-4a10-9987-3540f1b20af2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:38.199199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1203/2393] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:38.210339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3815ffc-f07a-46b5-a49f-c6c49041f4ab None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:38.212096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a3815ffc-f07a-46b5-a49f-c6c49041f4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:38.261127 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86bcb8-1b35-4a10-9987-3540f1b20af2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:38.262900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86bcb8-1b35-4a10-9987-3540f1b20af2 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:38.273115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1191/2394] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:03:38.293494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3815ffc-f07a-46b5-a49f-c6c49041f4ab None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:38.300020 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52262cf3-4b7c-4c8c-a2df-fc7acb072486 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:38.300669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52262cf3-4b7c-4c8c-a2df-fc7acb072486 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:38.300951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52262cf3-4b7c-4c8c-a2df-fc7acb072486 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:38.301227 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52262cf3-4b7c-4c8c-a2df-fc7acb072486 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:38.309739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52262cf3-4b7c-4c8c-a2df-fc7acb072486 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:38.311582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52262cf3-4b7c-4c8c-a2df-fc7acb072486 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:38.321329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1204/2395] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:03:38.435332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86bcb8-1b35-4a10-9987-3540f1b20af2 service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:38.501287 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86bcb8-1b35-4a10-9987-3540f1b20af2 service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:38.505964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:38.506917 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:38.507176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:38.507450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] PATH_INFO: `/v3/users/c1b4f67445db47c9bcdeb6f5df6e9705` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:38.508666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1b4f67445db47c9bcdeb6f5df6e9705)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:38.510598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:38.562512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1b4f67445db47c9bcdeb6f5df6e9705'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:38.563330 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1b4f67445db47c9bcdeb6f5df6e9705'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:38.576675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1b4f67445db47c9bcdeb6f5df6e9705'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:38.578207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1b4f67445db47c9bcdeb6f5df6e9705'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:38.587650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1b4f67445db47c9bcdeb6f5df6e9705'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:38.603230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1192/2396] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:38 2022] DELETE /identity/v3/users/c1b4f67445db47c9bcdeb6f5df6e9705 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:38.608079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c01f77-7ef9-4037-a8b1-2b77fe993857 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:38.608642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c01f77-7ef9-4037-a8b1-2b77fe993857 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:38.608732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c01f77-7ef9-4037-a8b1-2b77fe993857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:38.634491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-28c01f77-7ef9-4037-a8b1-2b77fe993857 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:38.635668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-28c01f77-7ef9-4037-a8b1-2b77fe993857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:38.700900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1205/2397] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:38.711833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:38.713460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:38.778483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a23132-63e6-49ba-83ae-d9a7fcb5ff2c admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:38.783173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279d5ff-b201-452a-bf1d-558bca046681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:38.783835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279d5ff-b201-452a-bf1d-558bca046681 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:38.784154 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279d5ff-b201-452a-bf1d-558bca046681 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:38.784387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279d5ff-b201-452a-bf1d-558bca046681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:38.794312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279d5ff-b201-452a-bf1d-558bca046681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:38.796229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279d5ff-b201-452a-bf1d-558bca046681 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:38.807095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1193/2398] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:39.100182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c01f77-7ef9-4037-a8b1-2b77fe993857 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:39.101662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-28c01f77-7ef9-4037-a8b1-2b77fe993857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:39.116807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c01f77-7ef9-4037-a8b1-2b77fe993857 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:39.122173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:39.123178 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:39.123549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:39.123915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] PATH_INFO: `/v3/projects/fc755d186d004ab3805c54d8f41cc982` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:39.125758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc755d186d004ab3805c54d8f41cc982)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:39.127896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:39.179539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc755d186d004ab3805c54d8f41cc982', 'request_id': 'req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.192972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc755d186d004ab3805c54d8f41cc982', 'request_id': 'req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.198027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc755d186d004ab3805c54d8f41cc982', 'request_id': 'req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.209570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] The token cache is being invalidate because project fc755d186d004ab3805c54d8f41cc982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:39.209848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.210497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.212066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1206/2399] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:39 2022] DELETE /identity/v3/projects/fc755d186d004ab3805c54d8f41cc982 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:39.216318 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279d5ff-b201-452a-bf1d-558bca046681 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:39.225594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279d5ff-b201-452a-bf1d-558bca046681 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:39.282240 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:39.283142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:39.283421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:39.283692 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] PATH_INFO: `/v3/users/1edb2918ae3b4256bb96a5067bfccdb3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:39.284885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1edb2918ae3b4256bb96a5067bfccdb3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:39.286870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:39.336604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1edb2918ae3b4256bb96a5067bfccdb3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.337460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1edb2918ae3b4256bb96a5067bfccdb3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.349635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1edb2918ae3b4256bb96a5067bfccdb3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.350293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1edb2918ae3b4256bb96a5067bfccdb3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.357334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29264782-8d5c-4a5f-a171-4ccc27d1a4f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1edb2918ae3b4256bb96a5067bfccdb3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.370402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1194/2400] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:39 2022] DELETE /identity/v3/users/1edb2918ae3b4256bb96a5067bfccdb3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:39.655235 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:39.658306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:39.672390 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79f4e9-1612-4541-aa60-4e74cc57b8fd admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:39.731720 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:39.732681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:39.732968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:39.733255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] PATH_INFO: `/v3/projects/058c95cb7bae4085962b74fcbb78ad63` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:39.735067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=058c95cb7bae4085962b74fcbb78ad63)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:39.737084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:39.783725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058c95cb7bae4085962b74fcbb78ad63', 'request_id': 'req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.790423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058c95cb7bae4085962b74fcbb78ad63', 'request_id': 'req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.799662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058c95cb7bae4085962b74fcbb78ad63', 'request_id': 'req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.811015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] The token cache is being invalidate because project 058c95cb7bae4085962b74fcbb78ad63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:39.811299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.812571 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a47f6ec7-e7a8-458f-ad55-c7259f6fcd65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:39.814148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1207/2401] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:39 2022] DELETE /identity/v3/projects/058c95cb7bae4085962b74fcbb78ad63 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:39.819918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024574a1-f24b-496c-af8a-d73d49f46f2b None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:39.820421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024574a1-f24b-496c-af8a-d73d49f46f2b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:39.820646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024574a1-f24b-496c-af8a-d73d49f46f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:39.840626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-024574a1-f24b-496c-af8a-d73d49f46f2b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:39.841830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-024574a1-f24b-496c-af8a-d73d49f46f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:39.919298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1195/2402] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:39.922326 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05117a6a-1819-4b38-bf2c-9d391ad9b731 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:39.922687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05117a6a-1819-4b38-bf2c-9d391ad9b731 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:39.923062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05117a6a-1819-4b38-bf2c-9d391ad9b731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:39.944139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-05117a6a-1819-4b38-bf2c-9d391ad9b731 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:39.945354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-05117a6a-1819-4b38-bf2c-9d391ad9b731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:39.955047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1208/2403] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:39.959051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024574a1-f24b-496c-af8a-d73d49f46f2b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:39.968829 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024574a1-f24b-496c-af8a-d73d49f46f2b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:39.978383 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ba195-7a20-4bbc-96d7-6988631bd228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:39.978531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ba195-7a20-4bbc-96d7-6988631bd228 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:39.978837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ba195-7a20-4bbc-96d7-6988631bd228 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:39.979562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ba195-7a20-4bbc-96d7-6988631bd228 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:39.979837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ba195-7a20-4bbc-96d7-6988631bd228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:39.980146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ba195-7a20-4bbc-96d7-6988631bd228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:39.981822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ba195-7a20-4bbc-96d7-6988631bd228 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:39.994775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1196/2404] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:03:40.000670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05117a6a-1819-4b38-bf2c-9d391ad9b731 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.009923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05117a6a-1819-4b38-bf2c-9d391ad9b731 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.015503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b32ee-0834-4f2b-a6d2-4398cd39f029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.016167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b32ee-0834-4f2b-a6d2-4398cd39f029 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.016451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b32ee-0834-4f2b-a6d2-4398cd39f029 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.016734 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b32ee-0834-4f2b-a6d2-4398cd39f029 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.017426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b32ee-0834-4f2b-a6d2-4398cd39f029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.019413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b32ee-0834-4f2b-a6d2-4398cd39f029 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.049583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1209/2405] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:39 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:40.050980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ba195-7a20-4bbc-96d7-6988631bd228 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.062409 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ba195-7a20-4bbc-96d7-6988631bd228 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.132434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23823c-00c4-4861-b5f1-d00d2e5cdbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.133215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23823c-00c4-4861-b5f1-d00d2e5cdbfb admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.133370 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23823c-00c4-4861-b5f1-d00d2e5cdbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.133646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23823c-00c4-4861-b5f1-d00d2e5cdbfb admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.134391 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23823c-00c4-4861-b5f1-d00d2e5cdbfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.139642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23823c-00c4-4861-b5f1-d00d2e5cdbfb admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.163518 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1197/2406] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:40 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:40.168005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b32ee-0834-4f2b-a6d2-4398cd39f029 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.181324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b32ee-0834-4f2b-a6d2-4398cd39f029 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.206308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393eb2b4-5aaf-49af-a4be-9056681b5adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.207034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393eb2b4-5aaf-49af-a4be-9056681b5adc admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.207324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393eb2b4-5aaf-49af-a4be-9056681b5adc admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.207600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393eb2b4-5aaf-49af-a4be-9056681b5adc admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.208336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393eb2b4-5aaf-49af-a4be-9056681b5adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.210120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393eb2b4-5aaf-49af-a4be-9056681b5adc admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.228145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1210/2407] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:40.228567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23823c-00c4-4861-b5f1-d00d2e5cdbfb admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.236434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23823c-00c4-4861-b5f1-d00d2e5cdbfb admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.241636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991a72c-1b5f-44dc-9a60-f12f568cb611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.242301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a72c-1b5f-44dc-9a60-f12f568cb611 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.242578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a72c-1b5f-44dc-9a60-f12f568cb611 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.242941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a72c-1b5f-44dc-9a60-f12f568cb611 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.243614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991a72c-1b5f-44dc-9a60-f12f568cb611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.245388 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991a72c-1b5f-44dc-9a60-f12f568cb611 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.262765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1198/2408] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:03:40.269640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393eb2b4-5aaf-49af-a4be-9056681b5adc admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.286479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393eb2b4-5aaf-49af-a4be-9056681b5adc admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.291946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11695f-8a57-4457-be48-0040cf21beeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.292856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11695f-8a57-4457-be48-0040cf21beeb admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.293139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11695f-8a57-4457-be48-0040cf21beeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.293428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11695f-8a57-4457-be48-0040cf21beeb admin admin] PATH_INFO: `/v3/projects/6422e212035342a1bcce6da7cf2427c7/users/5ead17b3fb904f91a5631967885eeddd/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.313276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11695f-8a57-4457-be48-0040cf21beeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6422e212035342a1bcce6da7cf2427c7, user_id=5ead17b3fb904f91a5631967885eeddd, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.316011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11695f-8a57-4457-be48-0040cf21beeb admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.330492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1211/2409] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:40 2022] PUT /identity/v3/projects/6422e212035342a1bcce6da7cf2427c7/users/5ead17b3fb904f91a5631967885eeddd/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:40.340828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024489d2-b47c-4666-a288-0c048be5ac15 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.341118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024489d2-b47c-4666-a288-0c048be5ac15 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.341411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024489d2-b47c-4666-a288-0c048be5ac15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.370982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-024489d2-b47c-4666-a288-0c048be5ac15 None None] MFA Rules not processed for user `5ead17b3fb904f91a5631967885eeddd`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:40.372502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-024489d2-b47c-4666-a288-0c048be5ac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:40.437588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1199/2410] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:40.442097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11695f-8a57-4457-be48-0040cf21beeb admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.503598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11695f-8a57-4457-be48-0040cf21beeb admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.508248 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1041e13-a298-4a93-8587-0630faa57e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.508896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1041e13-a298-4a93-8587-0630faa57e81 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.509210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1041e13-a298-4a93-8587-0630faa57e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.509443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1041e13-a298-4a93-8587-0630faa57e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.510132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1041e13-a298-4a93-8587-0630faa57e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.512084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1041e13-a298-4a93-8587-0630faa57e81 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.537877 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1212/2411] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:40 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:40.541965 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024489d2-b47c-4666-a288-0c048be5ac15 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.553545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024489d2-b47c-4666-a288-0c048be5ac15 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.609734 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10974cbd-7224-4441-991c-6dabc8584a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.610408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974cbd-7224-4441-991c-6dabc8584a8d admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.610688 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974cbd-7224-4441-991c-6dabc8584a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.610991 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974cbd-7224-4441-991c-6dabc8584a8d admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.611690 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10974cbd-7224-4441-991c-6dabc8584a8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.613612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10974cbd-7224-4441-991c-6dabc8584a8d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.633715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1200/2412] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:40 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:40.637780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1041e13-a298-4a93-8587-0630faa57e81 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.647921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1041e13-a298-4a93-8587-0630faa57e81 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.653036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1008d1b-66ab-4db9-b99d-b9f348170c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.653708 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1008d1b-66ab-4db9-b99d-b9f348170c9c admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.653993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1008d1b-66ab-4db9-b99d-b9f348170c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.654272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1008d1b-66ab-4db9-b99d-b9f348170c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.655033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1008d1b-66ab-4db9-b99d-b9f348170c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.657194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1008d1b-66ab-4db9-b99d-b9f348170c9c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.667815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1213/2413] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:40.673408 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10974cbd-7224-4441-991c-6dabc8584a8d admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.683527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10974cbd-7224-4441-991c-6dabc8584a8d admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.688195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a462e-ae54-411f-b1be-393a7b539fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.689069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a462e-ae54-411f-b1be-393a7b539fff admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.689376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a462e-ae54-411f-b1be-393a7b539fff admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.689651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a462e-ae54-411f-b1be-393a7b539fff admin admin] PATH_INFO: `/v3/projects/4a89e823d6084b7ea314993796a01169/users/96ebe1a77d014bafafcfef9c15ca05f8/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.704181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a462e-ae54-411f-b1be-393a7b539fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a89e823d6084b7ea314993796a01169, user_id=96ebe1a77d014bafafcfef9c15ca05f8, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.706931 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a462e-ae54-411f-b1be-393a7b539fff admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.720586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1201/2414] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:40 2022] PUT /identity/v3/projects/4a89e823d6084b7ea314993796a01169/users/96ebe1a77d014bafafcfef9c15ca05f8/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:40.724528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1008d1b-66ab-4db9-b99d-b9f348170c9c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.735007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1008d1b-66ab-4db9-b99d-b9f348170c9c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.789935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390a5d4-0b20-4280-8fc2-1b2486d1bbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.790522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a5d4-0b20-4280-8fc2-1b2486d1bbab admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.790928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a5d4-0b20-4280-8fc2-1b2486d1bbab admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.791210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a5d4-0b20-4280-8fc2-1b2486d1bbab admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.791931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390a5d4-0b20-4280-8fc2-1b2486d1bbab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.793533 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390a5d4-0b20-4280-8fc2-1b2486d1bbab admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.804293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1214/2415] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:40.809346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a462e-ae54-411f-b1be-393a7b539fff admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.819677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a462e-ae54-411f-b1be-393a7b539fff admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:40.824563 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cf021-4465-442d-a893-fde7e7243477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:40.825446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cf021-4465-442d-a893-fde7e7243477 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.825727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cf021-4465-442d-a893-fde7e7243477 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.826005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cf021-4465-442d-a893-fde7e7243477 admin admin] PATH_INFO: `/v3/projects/4a89e823d6084b7ea314993796a01169/users/96ebe1a77d014bafafcfef9c15ca05f8/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.828609 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616cf021-4465-442d-a893-fde7e7243477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a89e823d6084b7ea314993796a01169, user_id=96ebe1a77d014bafafcfef9c15ca05f8, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:40.831291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616cf021-4465-442d-a893-fde7e7243477 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:40.841758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1202/2416] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:40 2022] PUT /identity/v3/projects/4a89e823d6084b7ea314993796a01169/users/96ebe1a77d014bafafcfef9c15ca05f8/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:40.853908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ab0a9-7878-40a3-b658-71e1f4763004 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:40.854515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ab0a9-7878-40a3-b658-71e1f4763004 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:40.854848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ab0a9-7878-40a3-b658-71e1f4763004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:40.882154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-710ab0a9-7878-40a3-b658-71e1f4763004 None None] MFA Rules not processed for user `96ebe1a77d014bafafcfef9c15ca05f8`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:40.883409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-710ab0a9-7878-40a3-b658-71e1f4763004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:40.955328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1215/2417] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:40.966334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cf021-4465-442d-a893-fde7e7243477 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:40.968198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-616cf021-4465-442d-a893-fde7e7243477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:41.025909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cf021-4465-442d-a893-fde7e7243477 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:41.029575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb394980-839f-4c15-a7e7-b0e731b28789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:41.030239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394980-839f-4c15-a7e7-b0e731b28789 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:41.030894 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394980-839f-4c15-a7e7-b0e731b28789 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:41.031175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394980-839f-4c15-a7e7-b0e731b28789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:41.040525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb394980-839f-4c15-a7e7-b0e731b28789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:41.041901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb394980-839f-4c15-a7e7-b0e731b28789 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:41.052877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1203/2418] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:41.417813 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ab0a9-7878-40a3-b658-71e1f4763004 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:41.482168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ab0a9-7878-40a3-b658-71e1f4763004 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:41.487591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:41.488497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:41.488779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:41.489136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] PATH_INFO: `/v3/users/5ead17b3fb904f91a5631967885eeddd` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:41.490288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ead17b3fb904f91a5631967885eeddd)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:41.492245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:41.538273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ead17b3fb904f91a5631967885eeddd'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:41.550197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ead17b3fb904f91a5631967885eeddd'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:41.556693 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ead17b3fb904f91a5631967885eeddd'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:41.557391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ead17b3fb904f91a5631967885eeddd'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:41.564853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ead17b3fb904f91a5631967885eeddd'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:41.570898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1216/2419] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:41 2022] DELETE /identity/v3/users/5ead17b3fb904f91a5631967885eeddd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:41.575911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5c260-dc23-42eb-a043-064babdd2ed2 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:41.576189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5c260-dc23-42eb-a043-064babdd2ed2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:41.576452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5c260-dc23-42eb-a043-064babdd2ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:41.596689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c5f5c260-dc23-42eb-a043-064babdd2ed2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:41.597921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c5f5c260-dc23-42eb-a043-064babdd2ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:41.657132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1204/2420] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:41.667018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:41.668447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:41.726451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3624c-d370-46ee-ad4b-d595f8245bc8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:41.730135 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6113e0d-fabb-45d9-bf7b-bc69dc2b9400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:41.730825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6113e0d-fabb-45d9-bf7b-bc69dc2b9400 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:41.731104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6113e0d-fabb-45d9-bf7b-bc69dc2b9400 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:41.731380 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6113e0d-fabb-45d9-bf7b-bc69dc2b9400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:41.740513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6113e0d-fabb-45d9-bf7b-bc69dc2b9400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:41.741935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6113e0d-fabb-45d9-bf7b-bc69dc2b9400 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:41.751303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1217/2421] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:42.024302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5c260-dc23-42eb-a043-064babdd2ed2 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:42.025830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c5f5c260-dc23-42eb-a043-064babdd2ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:42.041384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5c260-dc23-42eb-a043-064babdd2ed2 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:42.048312 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:42.049254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:42.049649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:42.050043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] PATH_INFO: `/v3/projects/6422e212035342a1bcce6da7cf2427c7` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:42.051694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6422e212035342a1bcce6da7cf2427c7)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:42.053791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:42.100076 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6422e212035342a1bcce6da7cf2427c7', 'request_id': 'req-dcca3711-b708-486e-92fa-515e4849b86c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.106003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6422e212035342a1bcce6da7cf2427c7', 'request_id': 'req-dcca3711-b708-486e-92fa-515e4849b86c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.112040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6422e212035342a1bcce6da7cf2427c7', 'request_id': 'req-dcca3711-b708-486e-92fa-515e4849b86c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.125398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] The token cache is being invalidate because project 6422e212035342a1bcce6da7cf2427c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:42.125683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.126342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.127946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1205/2422] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:42 2022] DELETE /identity/v3/projects/6422e212035342a1bcce6da7cf2427c7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:42.132126 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6113e0d-fabb-45d9-bf7b-bc69dc2b9400 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:42.141948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6113e0d-fabb-45d9-bf7b-bc69dc2b9400 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:42.197797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:42.198663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:42.199050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:42.199327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] PATH_INFO: `/v3/users/96ebe1a77d014bafafcfef9c15ca05f8` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:42.200519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96ebe1a77d014bafafcfef9c15ca05f8)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:42.202557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:42.250669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96ebe1a77d014bafafcfef9c15ca05f8'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.262123 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96ebe1a77d014bafafcfef9c15ca05f8'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.269127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96ebe1a77d014bafafcfef9c15ca05f8'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.269803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96ebe1a77d014bafafcfef9c15ca05f8'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.279324 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-87ab2b83-ca00-447d-a78f-35445f7cd490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96ebe1a77d014bafafcfef9c15ca05f8'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.285951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1218/2423] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:42 2022] DELETE /identity/v3/users/96ebe1a77d014bafafcfef9c15ca05f8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:42.562091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:42.563827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:42.579651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca3711-b708-486e-92fa-515e4849b86c admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:42.652971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:42.653839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:42.654142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:42.654417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] PATH_INFO: `/v3/projects/4a89e823d6084b7ea314993796a01169` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:42.656406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a89e823d6084b7ea314993796a01169)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:42.658333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:42.707589 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a89e823d6084b7ea314993796a01169', 'request_id': 'req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.714052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a89e823d6084b7ea314993796a01169', 'request_id': 'req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.719996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a89e823d6084b7ea314993796a01169', 'request_id': 'req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.732603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] The token cache is being invalidate because project 4a89e823d6084b7ea314993796a01169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:42.733269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.733533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-7fdac0c9-b5ee-44c1-9c60-d88233498ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:42.735704 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1206/2424] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:42 2022] DELETE /identity/v3/projects/4a89e823d6084b7ea314993796a01169 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:42.740554 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88154596-77f9-4022-900e-47febdb9c6ae None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:42.741062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88154596-77f9-4022-900e-47febdb9c6ae None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:42.741192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88154596-77f9-4022-900e-47febdb9c6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:42.759488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-88154596-77f9-4022-900e-47febdb9c6ae None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:42.760681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-88154596-77f9-4022-900e-47febdb9c6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:42.821398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1219/2425] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:42.830104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb6020-ca23-43de-afab-8f5e73b1a6d6 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:42.830390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb6020-ca23-43de-afab-8f5e73b1a6d6 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:42.830664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb6020-ca23-43de-afab-8f5e73b1a6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:42.849738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-babb6020-ca23-43de-afab-8f5e73b1a6d6 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:42.850971 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-babb6020-ca23-43de-afab-8f5e73b1a6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:42.858787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1207/2426] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:42.863191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88154596-77f9-4022-900e-47febdb9c6ae None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:42.872914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88154596-77f9-4022-900e-47febdb9c6ae None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:42.877984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576788e-20fe-4d05-93b3-24fc43b92bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:42.878645 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576788e-20fe-4d05-93b3-24fc43b92bac admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:42.878993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576788e-20fe-4d05-93b3-24fc43b92bac admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:42.879268 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576788e-20fe-4d05-93b3-24fc43b92bac admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:42.879993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576788e-20fe-4d05-93b3-24fc43b92bac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:42.880298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576788e-20fe-4d05-93b3-24fc43b92bac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:42.881890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576788e-20fe-4d05-93b3-24fc43b92bac admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:42.893864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1220/2427] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:03:42.898333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb6020-ca23-43de-afab-8f5e73b1a6d6 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:42.907583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb6020-ca23-43de-afab-8f5e73b1a6d6 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:42.912427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afe4f0-9a63-4e89-ab47-8741bbeebf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:42.913091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afe4f0-9a63-4e89-ab47-8741bbeebf99 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:42.913372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afe4f0-9a63-4e89-ab47-8741bbeebf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:42.913654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afe4f0-9a63-4e89-ab47-8741bbeebf99 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:42.914646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19afe4f0-9a63-4e89-ab47-8741bbeebf99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:42.916603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19afe4f0-9a63-4e89-ab47-8741bbeebf99 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:42.944533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1208/2428] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:42 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:42.948844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576788e-20fe-4d05-93b3-24fc43b92bac admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:42.959739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576788e-20fe-4d05-93b3-24fc43b92bac admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.015217 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bddc0-54e5-4f00-9f45-073355333e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.015887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bddc0-54e5-4f00-9f45-073355333e28 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.016170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bddc0-54e5-4f00-9f45-073355333e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.016446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bddc0-54e5-4f00-9f45-073355333e28 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.017141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bddc0-54e5-4f00-9f45-073355333e28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.019093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bddc0-54e5-4f00-9f45-073355333e28 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.037501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1221/2429] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:42 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:43.041644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afe4f0-9a63-4e89-ab47-8741bbeebf99 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:43.053185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afe4f0-9a63-4e89-ab47-8741bbeebf99 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.057830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ebc9e-9ae7-4ff5-a20b-72c84dca78f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.058490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ebc9e-9ae7-4ff5-a20b-72c84dca78f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.058812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ebc9e-9ae7-4ff5-a20b-72c84dca78f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.059092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ebc9e-9ae7-4ff5-a20b-72c84dca78f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.059821 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ebc9e-9ae7-4ff5-a20b-72c84dca78f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.061414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ebc9e-9ae7-4ff5-a20b-72c84dca78f9 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.072375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1209/2430] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:43.076697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bddc0-54e5-4f00-9f45-073355333e28 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:43.086377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bddc0-54e5-4f00-9f45-073355333e28 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.091077 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea56018-6eba-4f4a-bc02-15793db54981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.091732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea56018-6eba-4f4a-bc02-15793db54981 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.092066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea56018-6eba-4f4a-bc02-15793db54981 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.092289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea56018-6eba-4f4a-bc02-15793db54981 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.093031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea56018-6eba-4f4a-bc02-15793db54981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.094637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea56018-6eba-4f4a-bc02-15793db54981 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.104653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1222/2431] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:43.109321 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ebc9e-9ae7-4ff5-a20b-72c84dca78f9 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:43.119471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ebc9e-9ae7-4ff5-a20b-72c84dca78f9 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.124278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7249c-7afe-4061-80d8-505c8a028300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.125610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7249c-7afe-4061-80d8-505c8a028300 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.125890 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7249c-7afe-4061-80d8-505c8a028300 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.126164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7249c-7afe-4061-80d8-505c8a028300 admin admin] PATH_INFO: `/v3/projects/e02d703292174436acb09e9bb8a6e193/users/622f145411824b459d1c37a8db8a7404/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.141220 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7249c-7afe-4061-80d8-505c8a028300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02d703292174436acb09e9bb8a6e193, user_id=622f145411824b459d1c37a8db8a7404, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.143983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7249c-7afe-4061-80d8-505c8a028300 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.158473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1210/2432] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:43 2022] PUT /identity/v3/projects/e02d703292174436acb09e9bb8a6e193/users/622f145411824b459d1c37a8db8a7404/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:43.170014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a076a-8621-4906-bc41-7dc8387c18c6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.170294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a076a-8621-4906-bc41-7dc8387c18c6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.170847 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a076a-8621-4906-bc41-7dc8387c18c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.197955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1f2a076a-8621-4906-bc41-7dc8387c18c6 None None] MFA Rules not processed for user `622f145411824b459d1c37a8db8a7404`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:43.199226 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1f2a076a-8621-4906-bc41-7dc8387c18c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:43.263603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1223/2433] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:43.268891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7249c-7afe-4061-80d8-505c8a028300 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:43.332847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7249c-7afe-4061-80d8-505c8a028300 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.338160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390ef9d-0922-4f09-8d05-66e6da018f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.338948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390ef9d-0922-4f09-8d05-66e6da018f86 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.339228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390ef9d-0922-4f09-8d05-66e6da018f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.339502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390ef9d-0922-4f09-8d05-66e6da018f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.340205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390ef9d-0922-4f09-8d05-66e6da018f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.342156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390ef9d-0922-4f09-8d05-66e6da018f86 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.370733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1211/2434] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:43 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:43.375671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a076a-8621-4906-bc41-7dc8387c18c6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:43.388626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a076a-8621-4906-bc41-7dc8387c18c6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.464676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036bee2-d8b0-4ed6-b13a-f7ae6118d0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.465624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036bee2-d8b0-4ed6-b13a-f7ae6118d0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.465907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036bee2-d8b0-4ed6-b13a-f7ae6118d0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.466179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036bee2-d8b0-4ed6-b13a-f7ae6118d0f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.467022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036bee2-d8b0-4ed6-b13a-f7ae6118d0f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.469464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036bee2-d8b0-4ed6-b13a-f7ae6118d0f0 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.493180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1224/2435] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:43 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:43.497919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390ef9d-0922-4f09-8d05-66e6da018f86 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:43.517637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390ef9d-0922-4f09-8d05-66e6da018f86 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.522437 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc9c4f-90ca-47dc-af1d-aca9f6b3582a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.523156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc9c4f-90ca-47dc-af1d-aca9f6b3582a admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.523599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc9c4f-90ca-47dc-af1d-aca9f6b3582a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.523980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc9c4f-90ca-47dc-af1d-aca9f6b3582a admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.525017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc9c4f-90ca-47dc-af1d-aca9f6b3582a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.527767 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc9c4f-90ca-47dc-af1d-aca9f6b3582a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.543655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1212/2436] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:43.548580 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036bee2-d8b0-4ed6-b13a-f7ae6118d0f0 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:43.561907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036bee2-d8b0-4ed6-b13a-f7ae6118d0f0 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.566895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d62b-235b-45ec-a58e-2cbf9c1a2b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.567899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d62b-235b-45ec-a58e-2cbf9c1a2b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.568147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d62b-235b-45ec-a58e-2cbf9c1a2b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.568425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d62b-235b-45ec-a58e-2cbf9c1a2b6d admin admin] PATH_INFO: `/v3/projects/2451e8a8ca42478bbb9ab3791ff7a77e/users/c295beaaa0a547ad8e6ec10dabd25dd1/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.589018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649d62b-235b-45ec-a58e-2cbf9c1a2b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2451e8a8ca42478bbb9ab3791ff7a77e, user_id=c295beaaa0a547ad8e6ec10dabd25dd1, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.591735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649d62b-235b-45ec-a58e-2cbf9c1a2b6d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.613299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1225/2437] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:43 2022] PUT /identity/v3/projects/2451e8a8ca42478bbb9ab3791ff7a77e/users/c295beaaa0a547ad8e6ec10dabd25dd1/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:43.620546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc9c4f-90ca-47dc-af1d-aca9f6b3582a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:43.632954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc9c4f-90ca-47dc-af1d-aca9f6b3582a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.701649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca61cb-90db-4844-acd4-3e1ddbbdb49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.702347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca61cb-90db-4844-acd4-3e1ddbbdb49a admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.702644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca61cb-90db-4844-acd4-3e1ddbbdb49a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.702992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca61cb-90db-4844-acd4-3e1ddbbdb49a admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.703716 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca61cb-90db-4844-acd4-3e1ddbbdb49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.705338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca61cb-90db-4844-acd4-3e1ddbbdb49a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.715878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1213/2438] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:43.721377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d62b-235b-45ec-a58e-2cbf9c1a2b6d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:43.731817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d62b-235b-45ec-a58e-2cbf9c1a2b6d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.739947 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26056df6-d06d-4387-a8ff-d6477b309d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.740850 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26056df6-d06d-4387-a8ff-d6477b309d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.741454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26056df6-d06d-4387-a8ff-d6477b309d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.741454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26056df6-d06d-4387-a8ff-d6477b309d2f admin admin] PATH_INFO: `/v3/projects/2451e8a8ca42478bbb9ab3791ff7a77e/users/c295beaaa0a547ad8e6ec10dabd25dd1/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.743603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26056df6-d06d-4387-a8ff-d6477b309d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2451e8a8ca42478bbb9ab3791ff7a77e, user_id=c295beaaa0a547ad8e6ec10dabd25dd1, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.746397 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26056df6-d06d-4387-a8ff-d6477b309d2f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.758118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1226/2439] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:43 2022] PUT /identity/v3/projects/2451e8a8ca42478bbb9ab3791ff7a77e/users/c295beaaa0a547ad8e6ec10dabd25dd1/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:43.769694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcece4f-ad32-41f8-b555-a0c532f0ff2d None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.770113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcece4f-ad32-41f8-b555-a0c532f0ff2d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.770453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcece4f-ad32-41f8-b555-a0c532f0ff2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.800423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-afcece4f-ad32-41f8-b555-a0c532f0ff2d None None] MFA Rules not processed for user `c295beaaa0a547ad8e6ec10dabd25dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:43.801632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-afcece4f-ad32-41f8-b555-a0c532f0ff2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:43.878475 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1214/2440] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:43.891026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26056df6-d06d-4387-a8ff-d6477b309d2f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:43.892741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-26056df6-d06d-4387-a8ff-d6477b309d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:43.951810 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26056df6-d06d-4387-a8ff-d6477b309d2f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:43.956423 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97a6f0-2135-4719-b028-5384c86cc0f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:43.957066 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97a6f0-2135-4719-b028-5384c86cc0f2 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:43.957420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97a6f0-2135-4719-b028-5384c86cc0f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:43.957730 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97a6f0-2135-4719-b028-5384c86cc0f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:43.970050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97a6f0-2135-4719-b028-5384c86cc0f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:43.971534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97a6f0-2135-4719-b028-5384c86cc0f2 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:43.981684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1227/2441] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:44.073631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcece4f-ad32-41f8-b555-a0c532f0ff2d None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:44.084092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcece4f-ad32-41f8-b555-a0c532f0ff2d None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:44.088104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bf25c-3832-4de2-83d0-c202526e7392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:44.088674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf25c-3832-4de2-83d0-c202526e7392 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:44.088955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf25c-3832-4de2-83d0-c202526e7392 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:44.089232 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf25c-3832-4de2-83d0-c202526e7392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:44.146116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bf25c-3832-4de2-83d0-c202526e7392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:44.147530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bf25c-3832-4de2-83d0-c202526e7392 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:44.156810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1215/2442] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:44.438091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97a6f0-2135-4719-b028-5384c86cc0f2 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:44.501775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97a6f0-2135-4719-b028-5384c86cc0f2 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:44.510931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:44.510931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:44.510931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:44.510931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] PATH_INFO: `/v3/users/622f145411824b459d1c37a8db8a7404` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:44.511412 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=622f145411824b459d1c37a8db8a7404)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:44.512760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:44.573942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '622f145411824b459d1c37a8db8a7404'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:44.587303 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '622f145411824b459d1c37a8db8a7404'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:44.601322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '622f145411824b459d1c37a8db8a7404'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:44.602084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '622f145411824b459d1c37a8db8a7404'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:44.621013 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '622f145411824b459d1c37a8db8a7404'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:44.635374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1228/2443] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:44 2022] DELETE /identity/v3/users/622f145411824b459d1c37a8db8a7404 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:44.638382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d168e-3972-4b86-97e3-7aefcc85b769 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:44.641268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d168e-3972-4b86-97e3-7aefcc85b769 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:44.641582 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d168e-3972-4b86-97e3-7aefcc85b769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:44.668015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:44.669576 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:44.684691 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a23d168e-3972-4b86-97e3-7aefcc85b769 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:44.686600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a23d168e-3972-4b86-97e3-7aefcc85b769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:44.749833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f54f90-f5da-435a-b391-24931826ec7c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:44.756409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:44.757586 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:44.758027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:44.758485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] PATH_INFO: `/v3/users/f1eaf82e62cc445ea8659fd2e8664dbf` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:44.760145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1eaf82e62cc445ea8659fd2e8664dbf)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:44.763115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:44.779924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1216/2444] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:44.791573 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d168e-3972-4b86-97e3-7aefcc85b769 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:44.793364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a23d168e-3972-4b86-97e3-7aefcc85b769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:44.825584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1eaf82e62cc445ea8659fd2e8664dbf'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:44.837993 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1eaf82e62cc445ea8659fd2e8664dbf'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:44.845515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1eaf82e62cc445ea8659fd2e8664dbf'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:44.846424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1eaf82e62cc445ea8659fd2e8664dbf'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:44.856263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7baa7f41-9529-41c9-a451-c1f03fa58425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1eaf82e62cc445ea8659fd2e8664dbf'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:44.859378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d168e-3972-4b86-97e3-7aefcc85b769 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:44.863767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1229/2445] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:44 2022] DELETE /identity/v3/users/f1eaf82e62cc445ea8659fd2e8664dbf => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:44.865101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcb70e-3a63-4a09-8cc1-aa51d6cad793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:44.865768 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcb70e-3a63-4a09-8cc1-aa51d6cad793 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:44.866048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcb70e-3a63-4a09-8cc1-aa51d6cad793 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:44.866325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcb70e-3a63-4a09-8cc1-aa51d6cad793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:44.868328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2adcb70e-3a63-4a09-8cc1-aa51d6cad793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:44.868627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328cfb3-9a35-44de-bfcb-111c872aa044 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:44.869280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328cfb3-9a35-44de-bfcb-111c872aa044 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:44.869780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328cfb3-9a35-44de-bfcb-111c872aa044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:44.884378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcb70e-3a63-4a09-8cc1-aa51d6cad793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:44.886535 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcb70e-3a63-4a09-8cc1-aa51d6cad793 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:44.893562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-7328cfb3-9a35-44de-bfcb-111c872aa044 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:44.894465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7328cfb3-9a35-44de-bfcb-111c872aa044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:44.902415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1217/2446] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:44.962492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1230/2447] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:44.973677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcb70e-3a63-4a09-8cc1-aa51d6cad793 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:44.975309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2adcb70e-3a63-4a09-8cc1-aa51d6cad793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:44.989201 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcb70e-3a63-4a09-8cc1-aa51d6cad793 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:45.050198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927c696-0637-4ac0-a781-a71266ca042b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:45.050898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927c696-0637-4ac0-a781-a71266ca042b service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:45.051180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927c696-0637-4ac0-a781-a71266ca042b service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:45.051457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927c696-0637-4ac0-a781-a71266ca042b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:45.062890 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927c696-0637-4ac0-a781-a71266ca042b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:45.064642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927c696-0637-4ac0-a781-a71266ca042b service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:45.078324 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1218/2448] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:45.328157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328cfb3-9a35-44de-bfcb-111c872aa044 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:45.330009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7328cfb3-9a35-44de-bfcb-111c872aa044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:45.347848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328cfb3-9a35-44de-bfcb-111c872aa044 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:45.354569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:45.355682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:45.356099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:45.356582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] PATH_INFO: `/v3/projects/e02d703292174436acb09e9bb8a6e193` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:45.358628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02d703292174436acb09e9bb8a6e193)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:45.361787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:45.420169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02d703292174436acb09e9bb8a6e193', 'request_id': 'req-c14cb37a-51c3-4933-83de-f88255c4e0ce'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.426602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02d703292174436acb09e9bb8a6e193', 'request_id': 'req-c14cb37a-51c3-4933-83de-f88255c4e0ce'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.435092 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02d703292174436acb09e9bb8a6e193', 'request_id': 'req-c14cb37a-51c3-4933-83de-f88255c4e0ce'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.447677 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] The token cache is being invalidate because project e02d703292174436acb09e9bb8a6e193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:45.447980 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.448761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.450625 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1231/2449] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:45 2022] DELETE /identity/v3/projects/e02d703292174436acb09e9bb8a6e193 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:45.455456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927c696-0637-4ac0-a781-a71266ca042b service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:45.465935 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927c696-0637-4ac0-a781-a71266ca042b service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:45.526182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:45.527633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:45.541458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:45.543129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:45.543574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:45.543864 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] PATH_INFO: `/v3/users/c295beaaa0a547ad8e6ec10dabd25dd1` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:45.545164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c295beaaa0a547ad8e6ec10dabd25dd1)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:45.547566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:45.600944 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c295beaaa0a547ad8e6ec10dabd25dd1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.601673 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c295beaaa0a547ad8e6ec10dabd25dd1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.605295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cb37a-51c3-4933-83de-f88255c4e0ce admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:45.611167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:45.612294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:45.612749 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:45.613033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] PATH_INFO: `/v3/projects/1fcb7d94e2ba453590ff33e773fd990c` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:45.614405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fcb7d94e2ba453590ff33e773fd990c)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:45.615822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c295beaaa0a547ad8e6ec10dabd25dd1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.616364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:45.616827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c295beaaa0a547ad8e6ec10dabd25dd1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.626539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c295beaaa0a547ad8e6ec10dabd25dd1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.640732 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1219/2450] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:45 2022] DELETE /identity/v3/users/c295beaaa0a547ad8e6ec10dabd25dd1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:45.674164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fcb7d94e2ba453590ff33e773fd990c', 'request_id': 'req-854593ff-0453-4e3e-a800-4cfa9b00e410'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.681342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fcb7d94e2ba453590ff33e773fd990c', 'request_id': 'req-854593ff-0453-4e3e-a800-4cfa9b00e410'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.689526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fcb7d94e2ba453590ff33e773fd990c', 'request_id': 'req-854593ff-0453-4e3e-a800-4cfa9b00e410'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.701391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] The token cache is being invalidate because project 1fcb7d94e2ba453590ff33e773fd990c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:45.701903 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.703029 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.704833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1232/2451] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:45 2022] DELETE /identity/v3/projects/1fcb7d94e2ba453590ff33e773fd990c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:45.708874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:45.710471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:45.786636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9af61a-9fe4-4f21-b40e-e8cf2f013804 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:45.851574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:45.852789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:45.852955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:45.854048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] PATH_INFO: `/v3/users/d2ed0de1c5b447ac8fc162221101a007` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:45.854729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ed0de1c5b447ac8fc162221101a007)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:45.856751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:45.930616 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ed0de1c5b447ac8fc162221101a007'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.931893 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ed0de1c5b447ac8fc162221101a007'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.954009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ed0de1c5b447ac8fc162221101a007'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.954954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ed0de1c5b447ac8fc162221101a007'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.964106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ed0de1c5b447ac8fc162221101a007'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:45.986871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1220/2452] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:45 2022] DELETE /identity/v3/users/d2ed0de1c5b447ac8fc162221101a007 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:46.003343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.005228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.018884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854593ff-0453-4e3e-a800-4cfa9b00e410 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.030641 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.032235 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.110832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.111742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.112246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.112727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] PATH_INFO: `/v3/users/10d7a3891aa244508dd55484ebb235cb` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.115460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10d7a3891aa244508dd55484ebb235cb)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.118437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.146030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64b3cc-a377-4859-9bc9-f5552d7cdba3 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.152840 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.153841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.154192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.154586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] PATH_INFO: `/v3/projects/2451e8a8ca42478bbb9ab3791ff7a77e` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.156346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2451e8a8ca42478bbb9ab3791ff7a77e)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.158369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.183660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10d7a3891aa244508dd55484ebb235cb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.203730 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10d7a3891aa244508dd55484ebb235cb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.212358 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10d7a3891aa244508dd55484ebb235cb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.213255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10d7a3891aa244508dd55484ebb235cb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.214189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2451e8a8ca42478bbb9ab3791ff7a77e', 'request_id': 'req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.223307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-04799973-5349-43dc-be4d-3d5f3b39ee85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10d7a3891aa244508dd55484ebb235cb'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.225028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2451e8a8ca42478bbb9ab3791ff7a77e', 'request_id': 'req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.230540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1233/2453] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:46 2022] DELETE /identity/v3/users/10d7a3891aa244508dd55484ebb235cb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:46.231311 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2451e8a8ca42478bbb9ab3791ff7a77e', 'request_id': 'req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.235699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998011f-0685-47db-8992-14774daafede None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.235988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998011f-0685-47db-8992-14774daafede None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.236440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998011f-0685-47db-8992-14774daafede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.246051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] The token cache is being invalidate because project 2451e8a8ca42478bbb9ab3791ff7a77e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:46.246333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.247115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ecbd6b98-a953-41c7-a2b7-a9c4c119b9ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.248883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1221/2454] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:46 2022] DELETE /identity/v3/projects/2451e8a8ca42478bbb9ab3791ff7a77e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:46.253636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11375724-1e05-40d4-b88d-1101e1adafc2 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.253916 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11375724-1e05-40d4-b88d-1101e1adafc2 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.254193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11375724-1e05-40d4-b88d-1101e1adafc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.257566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-e998011f-0685-47db-8992-14774daafede None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:46.259230 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e998011f-0685-47db-8992-14774daafede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.274856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-11375724-1e05-40d4-b88d-1101e1adafc2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:46.276259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-11375724-1e05-40d4-b88d-1101e1adafc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.336954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1234/2455] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:46.347776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1222/2456] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:46.351413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998011f-0685-47db-8992-14774daafede None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.353035 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e998011f-0685-47db-8992-14774daafede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.356743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a43845-9d9b-41d2-87b2-286f91219dcb None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.357262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a43845-9d9b-41d2-87b2-286f91219dcb None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.357877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a43845-9d9b-41d2-87b2-286f91219dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.380386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-d0a43845-9d9b-41d2-87b2-286f91219dcb None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:46.381955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-d0a43845-9d9b-41d2-87b2-286f91219dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.391778 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1223/2457] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:46.396291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a43845-9d9b-41d2-87b2-286f91219dcb None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.407208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a43845-9d9b-41d2-87b2-286f91219dcb None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.413396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38f847-c9ab-4983-b551-4f8a1c6ab5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.414202 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38f847-c9ab-4983-b551-4f8a1c6ab5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.414474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38f847-c9ab-4983-b551-4f8a1c6ab5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.414801 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38f847-c9ab-4983-b551-4f8a1c6ab5b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.415715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38f847-c9ab-4983-b551-4f8a1c6ab5b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:46.416198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38f847-c9ab-4983-b551-4f8a1c6ab5b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.418156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38f847-c9ab-4983-b551-4f8a1c6ab5b7 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.433900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1224/2458] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:03:46.434858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38f847-c9ab-4983-b551-4f8a1c6ab5b7 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.437457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-de38f847-c9ab-4983-b551-4f8a1c6ab5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.439815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998011f-0685-47db-8992-14774daafede None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.450170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf7b42-8357-49e6-a54d-965fa090f3ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.451170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7b42-8357-49e6-a54d-965fa090f3ac service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.451453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7b42-8357-49e6-a54d-965fa090f3ac service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.451729 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7b42-8357-49e6-a54d-965fa090f3ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.465828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38f847-c9ab-4983-b551-4f8a1c6ab5b7 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.467482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf7b42-8357-49e6-a54d-965fa090f3ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.469329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf7b42-8357-49e6-a54d-965fa090f3ac service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.471067 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.471572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.472073 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] PATH_INFO: `/v3/projects/7951f55a395d4ef1b264dae366b31900` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.474316 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7951f55a395d4ef1b264dae366b31900)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.478135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.480841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1235/2459] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:03:46.481830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf7b42-8357-49e6-a54d-965fa090f3ac service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.492791 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf7b42-8357-49e6-a54d-965fa090f3ac service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.497837 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a5c04-8d83-4e28-8c01-617baa8b6f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.498545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a5c04-8d83-4e28-8c01-617baa8b6f45 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.498860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a5c04-8d83-4e28-8c01-617baa8b6f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.499136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a5c04-8d83-4e28-8c01-617baa8b6f45 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.499892 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a5c04-8d83-4e28-8c01-617baa8b6f45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.501908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a5c04-8d83-4e28-8c01-617baa8b6f45 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.537042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1236/2460] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:46 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:46.541237 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a5c04-8d83-4e28-8c01-617baa8b6f45 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.541694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7951f55a395d4ef1b264dae366b31900', 'request_id': 'req-ec9331fd-89c8-497f-abca-7424259fbab3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.551651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7951f55a395d4ef1b264dae366b31900', 'request_id': 'req-ec9331fd-89c8-497f-abca-7424259fbab3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.561964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7951f55a395d4ef1b264dae366b31900', 'request_id': 'req-ec9331fd-89c8-497f-abca-7424259fbab3'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.589931 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] The token cache is being invalidate because project 7951f55a395d4ef1b264dae366b31900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:46.590327 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.591912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-ec9331fd-89c8-497f-abca-7424259fbab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:46.594458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1225/2461] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:46 2022] DELETE /identity/v3/projects/7951f55a395d4ef1b264dae366b31900 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:46.600858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579effc1-4e1e-46f6-9746-79c9f3519d98 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.601239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579effc1-4e1e-46f6-9746-79c9f3519d98 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.601443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579effc1-4e1e-46f6-9746-79c9f3519d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.631229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-579effc1-4e1e-46f6-9746-79c9f3519d98 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:46.633198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-579effc1-4e1e-46f6-9746-79c9f3519d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.642526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a5c04-8d83-4e28-8c01-617baa8b6f45 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.649809 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1226/2462] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:46.651460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa1b39-03c3-4ae0-89b1-7c921e0b11a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.652676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa1b39-03c3-4ae0-89b1-7c921e0b11a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.653184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa1b39-03c3-4ae0-89b1-7c921e0b11a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.653914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa1b39-03c3-4ae0-89b1-7c921e0b11a8 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.658016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50797131-61fc-446a-ac06-ac1761c2cf0b None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.658458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50797131-61fc-446a-ac06-ac1761c2cf0b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.658796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50797131-61fc-446a-ac06-ac1761c2cf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.659297 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa1b39-03c3-4ae0-89b1-7c921e0b11a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.662168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa1b39-03c3-4ae0-89b1-7c921e0b11a8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.686135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-50797131-61fc-446a-ac06-ac1761c2cf0b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:46.688284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1237/2463] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:46 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:46.688972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-50797131-61fc-446a-ac06-ac1761c2cf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.692460 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa1b39-03c3-4ae0-89b1-7c921e0b11a8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.694617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0aaa1b39-03c3-4ae0-89b1-7c921e0b11a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.700621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1227/2464] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:46.705940 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50797131-61fc-446a-ac06-ac1761c2cf0b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.714962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa1b39-03c3-4ae0-89b1-7c921e0b11a8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.715499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50797131-61fc-446a-ac06-ac1761c2cf0b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.721942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0534d1-ac84-4856-ab2e-22be57e62c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.722694 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0534d1-ac84-4856-ab2e-22be57e62c4e admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.723097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0534d1-ac84-4856-ab2e-22be57e62c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.723452 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad2cc-6882-4065-86c2-5833a2aa0708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.723595 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0534d1-ac84-4856-ab2e-22be57e62c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.724128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad2cc-6882-4065-86c2-5833a2aa0708 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.724407 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad2cc-6882-4065-86c2-5833a2aa0708 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.724683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad2cc-6882-4065-86c2-5833a2aa0708 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.724804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0534d1-ac84-4856-ab2e-22be57e62c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.725421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ad2cc-6882-4065-86c2-5833a2aa0708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:46.725726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ad2cc-6882-4065-86c2-5833a2aa0708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.726331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0534d1-ac84-4856-ab2e-22be57e62c4e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.727937 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ad2cc-6882-4065-86c2-5833a2aa0708 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.746344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1228/2465] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:03:46.747096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1238/2466] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:03:46.752341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad2cc-6882-4065-86c2-5833a2aa0708 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.753769 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0534d1-ac84-4856-ab2e-22be57e62c4e admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.767079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad2cc-6882-4065-86c2-5833a2aa0708 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.773055 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e37bbc-1b2b-4b17-b7d6-1307aa28177c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.773558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0534d1-ac84-4856-ab2e-22be57e62c4e admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.773765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e37bbc-1b2b-4b17-b7d6-1307aa28177c admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.774029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e37bbc-1b2b-4b17-b7d6-1307aa28177c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.774290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e37bbc-1b2b-4b17-b7d6-1307aa28177c admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.775052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e37bbc-1b2b-4b17-b7d6-1307aa28177c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.777039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e37bbc-1b2b-4b17-b7d6-1307aa28177c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.790213 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d632e-034c-47b6-896f-b22554b19d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.790983 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d632e-034c-47b6-896f-b22554b19d4b admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.791264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d632e-034c-47b6-896f-b22554b19d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.791687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d632e-034c-47b6-896f-b22554b19d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.792629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d632e-034c-47b6-896f-b22554b19d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.794288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d632e-034c-47b6-896f-b22554b19d4b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.808403 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1239/2467] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:46.813158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d632e-034c-47b6-896f-b22554b19d4b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.818383 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1229/2468] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:46 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:46.824142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e37bbc-1b2b-4b17-b7d6-1307aa28177c admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.825949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d632e-034c-47b6-896f-b22554b19d4b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.915987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e37bbc-1b2b-4b17-b7d6-1307aa28177c admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:46.917722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415dce-f947-4599-93ba-05d3de6f15e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.918689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415dce-f947-4599-93ba-05d3de6f15e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.919075 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415dce-f947-4599-93ba-05d3de6f15e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.919621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c415dce-f947-4599-93ba-05d3de6f15e8 admin admin] PATH_INFO: `/v3/projects/b85eeb0a7ee0408ca90bb2e4c23f0543/users/2f9db02e76b2432d8cef24abd2570c83/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.922912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbed4aa-a170-48c9-985c-061d8cebe7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:46.923727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed4aa-a170-48c9-985c-061d8cebe7c0 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:46.924057 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed4aa-a170-48c9-985c-061d8cebe7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:46.924332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed4aa-a170-48c9-985c-061d8cebe7c0 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:46.925019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbed4aa-a170-48c9-985c-061d8cebe7c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.926981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbed4aa-a170-48c9-985c-061d8cebe7c0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.935608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c415dce-f947-4599-93ba-05d3de6f15e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b85eeb0a7ee0408ca90bb2e4c23f0543, user_id=2f9db02e76b2432d8cef24abd2570c83, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:46.938247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c415dce-f947-4599-93ba-05d3de6f15e8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:46.950681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1230/2469] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:46 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:46.951834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbed4aa-a170-48c9-985c-061d8cebe7c0 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.954079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1240/2470] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:46 2022] PUT /identity/v3/projects/b85eeb0a7ee0408ca90bb2e4c23f0543/users/2f9db02e76b2432d8cef24abd2570c83/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:46.954394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-3bbed4aa-a170-48c9-985c-061d8cebe7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:46.959484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415dce-f947-4599-93ba-05d3de6f15e8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:46.969473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbed4aa-a170-48c9-985c-061d8cebe7c0 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.026265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c415dce-f947-4599-93ba-05d3de6f15e8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.032379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64987b-2639-4f1a-a0c6-f4c37ae7db70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.033056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64987b-2639-4f1a-a0c6-f4c37ae7db70 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.033334 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64987b-2639-4f1a-a0c6-f4c37ae7db70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.033610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64987b-2639-4f1a-a0c6-f4c37ae7db70 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.034337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64987b-2639-4f1a-a0c6-f4c37ae7db70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.036009 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64987b-2639-4f1a-a0c6-f4c37ae7db70 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.036363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.041526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.041526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.041526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] PATH_INFO: `/v3/projects/9eb67099abec43c59e2aa5fe2f0920ab` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.041526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb67099abec43c59e2aa5fe2f0920ab)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.041526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.047479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1241/2471] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:47.048937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5360-f45c-4d2a-9e20-35944d8a05b0 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.049298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5360-f45c-4d2a-9e20-35944d8a05b0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.049637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5360-f45c-4d2a-9e20-35944d8a05b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.080330 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2d3a5360-f45c-4d2a-9e20-35944d8a05b0 None None] MFA Rules not processed for user `2f9db02e76b2432d8cef24abd2570c83`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:47.081560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2d3a5360-f45c-4d2a-9e20-35944d8a05b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:47.099816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb67099abec43c59e2aa5fe2f0920ab', 'request_id': 'req-0f513850-76ed-4c01-ab1b-568dd4b301d1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:47.106360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb67099abec43c59e2aa5fe2f0920ab', 'request_id': 'req-0f513850-76ed-4c01-ab1b-568dd4b301d1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:47.112236 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb67099abec43c59e2aa5fe2f0920ab', 'request_id': 'req-0f513850-76ed-4c01-ab1b-568dd4b301d1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:47.128939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] The token cache is being invalidate because project 9eb67099abec43c59e2aa5fe2f0920ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:47.130743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:47.134875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:47.137525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1231/2472] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:46 2022] DELETE /identity/v3/projects/9eb67099abec43c59e2aa5fe2f0920ab => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:47.138458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.140525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:47.158836 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1242/2473] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:47.160147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7a954-96a2-4220-b389-c6307cc36b01 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.160436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7a954-96a2-4220-b389-c6307cc36b01 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.160714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7a954-96a2-4220-b389-c6307cc36b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.180333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-ada7a954-96a2-4220-b389-c6307cc36b01 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:47.181593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ada7a954-96a2-4220-b389-c6307cc36b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:47.208453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f513850-76ed-4c01-ab1b-568dd4b301d1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.213753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246025eb-ec2a-4c86-a298-d213431a13ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.214515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246025eb-ec2a-4c86-a298-d213431a13ca admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.214904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246025eb-ec2a-4c86-a298-d213431a13ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.215179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246025eb-ec2a-4c86-a298-d213431a13ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.215905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246025eb-ec2a-4c86-a298-d213431a13ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.217524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246025eb-ec2a-4c86-a298-d213431a13ca admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.227807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1232/2474] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:47.229008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246025eb-ec2a-4c86-a298-d213431a13ca admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.231267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-246025eb-ec2a-4c86-a298-d213431a13ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:47.243678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1243/2475] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:47.244523 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7a954-96a2-4220-b389-c6307cc36b01 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.244689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246025eb-ec2a-4c86-a298-d213431a13ca admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.256522 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346924-26d9-4541-ae83-c9f4718dc785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.257584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346924-26d9-4541-ae83-c9f4718dc785 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.258028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346924-26d9-4541-ae83-c9f4718dc785 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.258471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346924-26d9-4541-ae83-c9f4718dc785 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.259699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a346924-26d9-4541-ae83-c9f4718dc785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.262612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a346924-26d9-4541-ae83-c9f4718dc785 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.265653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7a954-96a2-4220-b389-c6307cc36b01 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.270943 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d1f1a-ca3a-4e9e-8406-336341550f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.271876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d1f1a-ca3a-4e9e-8406-336341550f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.272158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d1f1a-ca3a-4e9e-8406-336341550f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.272430 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d1f1a-ca3a-4e9e-8406-336341550f4e admin admin] PATH_INFO: `/v3/projects/93dfe640b38f45f79fd2315cd7463f6b/users/cc88176d94bc498588e1702e4c8b1370/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.292214 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d1f1a-ca3a-4e9e-8406-336341550f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93dfe640b38f45f79fd2315cd7463f6b, user_id=cc88176d94bc498588e1702e4c8b1370, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.294921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d1f1a-ca3a-4e9e-8406-336341550f4e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.295523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1233/2476] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:47.297383 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa66ed0-e9d7-416b-b154-67017776c1bf None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.297735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa66ed0-e9d7-416b-b154-67017776c1bf None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.298109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa66ed0-e9d7-416b-b154-67017776c1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.313002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1244/2477] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:47 2022] PUT /identity/v3/projects/93dfe640b38f45f79fd2315cd7463f6b/users/cc88176d94bc498588e1702e4c8b1370/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:47.314902 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d1f1a-ca3a-4e9e-8406-336341550f4e admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.319753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-9aa66ed0-e9d7-416b-b154-67017776c1bf None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:47.321040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9aa66ed0-e9d7-416b-b154-67017776c1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:47.377862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d1f1a-ca3a-4e9e-8406-336341550f4e admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.380340 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1234/2478] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:47.381645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bc5c6-81ce-4a14-b191-29f1db758ff3 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.381934 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bc5c6-81ce-4a14-b191-29f1db758ff3 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.382212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bc5c6-81ce-4a14-b191-29f1db758ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.382758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fe932-bee1-41ec-b17b-d3388e200bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.383494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fe932-bee1-41ec-b17b-d3388e200bac admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.384074 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fe932-bee1-41ec-b17b-d3388e200bac admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.384207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fe932-bee1-41ec-b17b-d3388e200bac admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.384926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fe932-bee1-41ec-b17b-d3388e200bac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.387672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fe932-bee1-41ec-b17b-d3388e200bac admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.411544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1245/2479] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:47.411753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f69bc5c6-81ce-4a14-b191-29f1db758ff3 None None] MFA Rules not processed for user `cc88176d94bc498588e1702e4c8b1370`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:47.411899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fe932-bee1-41ec-b17b-d3388e200bac admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.412004 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f69bc5c6-81ce-4a14-b191-29f1db758ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:47.419529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fe932-bee1-41ec-b17b-d3388e200bac admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.425249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dec4dd-c3c8-46c9-ad95-2e90b89a2372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.426077 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dec4dd-c3c8-46c9-ad95-2e90b89a2372 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.426359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dec4dd-c3c8-46c9-ad95-2e90b89a2372 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.426637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dec4dd-c3c8-46c9-ad95-2e90b89a2372 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.427455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dec4dd-c3c8-46c9-ad95-2e90b89a2372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:47.427763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dec4dd-c3c8-46c9-ad95-2e90b89a2372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.429438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dec4dd-c3c8-46c9-ad95-2e90b89a2372 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.443095 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1246/2480] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:03:47.447475 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dec4dd-c3c8-46c9-ad95-2e90b89a2372 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.457643 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dec4dd-c3c8-46c9-ad95-2e90b89a2372 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.462428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e442285-776f-4d07-80f3-c067c33de60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.463051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e442285-776f-4d07-80f3-c067c33de60b admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.463329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e442285-776f-4d07-80f3-c067c33de60b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.463599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e442285-776f-4d07-80f3-c067c33de60b admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.464305 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e442285-776f-4d07-80f3-c067c33de60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.465883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e442285-776f-4d07-80f3-c067c33de60b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.477120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1247/2481] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:47.479736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e442285-776f-4d07-80f3-c067c33de60b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.480957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1235/2482] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:47.482082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bc5c6-81ce-4a14-b191-29f1db758ff3 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.490534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e442285-776f-4d07-80f3-c067c33de60b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.494574 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bc5c6-81ce-4a14-b191-29f1db758ff3 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.495522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ddaa7-0eac-414d-b616-15f296a4689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.496189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ddaa7-0eac-414d-b616-15f296a4689d admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.496474 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ddaa7-0eac-414d-b616-15f296a4689d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.496750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ddaa7-0eac-414d-b616-15f296a4689d admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.497570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ddaa7-0eac-414d-b616-15f296a4689d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.499767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ddaa7-0eac-414d-b616-15f296a4689d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.503008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f97b0-05c0-4e42-b63f-ad60568e902a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.504130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f97b0-05c0-4e42-b63f-ad60568e902a admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.504668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f97b0-05c0-4e42-b63f-ad60568e902a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.505096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f97b0-05c0-4e42-b63f-ad60568e902a admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.506483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f97b0-05c0-4e42-b63f-ad60568e902a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.509697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f97b0-05c0-4e42-b63f-ad60568e902a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.525540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1236/2483] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:47.526567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f97b0-05c0-4e42-b63f-ad60568e902a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.534781 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1248/2484] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:47.536122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ddaa7-0eac-414d-b616-15f296a4689d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.540583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f97b0-05c0-4e42-b63f-ad60568e902a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.601699 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bb904-3fa4-462f-8bab-32f100495864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.602263 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bb904-3fa4-462f-8bab-32f100495864 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.602538 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bb904-3fa4-462f-8bab-32f100495864 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.602870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bb904-3fa4-462f-8bab-32f100495864 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.603549 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bb904-3fa4-462f-8bab-32f100495864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.605511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bb904-3fa4-462f-8bab-32f100495864 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.612598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ddaa7-0eac-414d-b616-15f296a4689d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.618329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939fe6f-7829-4e26-9732-0c0d8791ee30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.619288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939fe6f-7829-4e26-9732-0c0d8791ee30 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.619575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939fe6f-7829-4e26-9732-0c0d8791ee30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.621920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939fe6f-7829-4e26-9732-0c0d8791ee30 admin admin] PATH_INFO: `/v3/projects/b886bb95424c43b98f23a94de334b3fb/users/90491414026748558450f28759ea6d41/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.638293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1237/2485] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:47.638539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939fe6f-7829-4e26-9732-0c0d8791ee30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b886bb95424c43b98f23a94de334b3fb, user_id=90491414026748558450f28759ea6d41, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.638642 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bb904-3fa4-462f-8bab-32f100495864 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.638762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939fe6f-7829-4e26-9732-0c0d8791ee30 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.652697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1249/2486] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:47 2022] PUT /identity/v3/projects/b886bb95424c43b98f23a94de334b3fb/users/90491414026748558450f28759ea6d41/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:47.653516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939fe6f-7829-4e26-9732-0c0d8791ee30 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.716006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bb904-3fa4-462f-8bab-32f100495864 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.719339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939fe6f-7829-4e26-9732-0c0d8791ee30 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.725444 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07c428-1a43-4f3f-9871-642689efd6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.726120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c428-1a43-4f3f-9871-642689efd6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.726248 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bd68e-ac8f-4c70-a09b-75062a06e8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.726772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c428-1a43-4f3f-9871-642689efd6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.726951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bd68e-ac8f-4c70-a09b-75062a06e8ec admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.727068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c428-1a43-4f3f-9871-642689efd6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.727303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bd68e-ac8f-4c70-a09b-75062a06e8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.727620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bd68e-ac8f-4c70-a09b-75062a06e8ec admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.727787 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07c428-1a43-4f3f-9871-642689efd6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.728457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bd68e-ac8f-4c70-a09b-75062a06e8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.729725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07c428-1a43-4f3f-9871-642689efd6c9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.730676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bd68e-ac8f-4c70-a09b-75062a06e8ec admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.755328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1250/2487] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:47.755328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d930188-66f7-4be2-9fa7-9bb6ec0e7294 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.755328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d930188-66f7-4be2-9fa7-9bb6ec0e7294 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.755328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d930188-66f7-4be2-9fa7-9bb6ec0e7294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.755794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1238/2488] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/users => generated 279 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:47.758911 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bd68e-ac8f-4c70-a09b-75062a06e8ec admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.769541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bd68e-ac8f-4c70-a09b-75062a06e8ec admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.774817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562520bb-f61f-42fc-901b-6a7cfdfa26c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.775481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562520bb-f61f-42fc-901b-6a7cfdfa26c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.775758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562520bb-f61f-42fc-901b-6a7cfdfa26c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.776027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562520bb-f61f-42fc-901b-6a7cfdfa26c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.776798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562520bb-f61f-42fc-901b-6a7cfdfa26c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.778432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562520bb-f61f-42fc-901b-6a7cfdfa26c0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.788492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6d930188-66f7-4be2-9fa7-9bb6ec0e7294 None None] MFA Rules not processed for user `90491414026748558450f28759ea6d41`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:47.789720 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6d930188-66f7-4be2-9fa7-9bb6ec0e7294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:47.796560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1239/2489] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:47.800093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562520bb-f61f-42fc-901b-6a7cfdfa26c0 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.810438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562520bb-f61f-42fc-901b-6a7cfdfa26c0 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.816342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1725c2-fe5c-4634-b3f2-7f4dac9e1d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.817061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1725c2-fe5c-4634-b3f2-7f4dac9e1d09 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.817347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1725c2-fe5c-4634-b3f2-7f4dac9e1d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.817676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1725c2-fe5c-4634-b3f2-7f4dac9e1d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.818455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1725c2-fe5c-4634-b3f2-7f4dac9e1d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.820189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1725c2-fe5c-4634-b3f2-7f4dac9e1d09 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.831089 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1240/2490] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:47.831918 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1725c2-fe5c-4634-b3f2-7f4dac9e1d09 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.844819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1725c2-fe5c-4634-b3f2-7f4dac9e1d09 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.852947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90477d0b-9c13-4832-bfe3-9008682c24d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.854383 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90477d0b-9c13-4832-bfe3-9008682c24d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.854870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90477d0b-9c13-4832-bfe3-9008682c24d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.855345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90477d0b-9c13-4832-bfe3-9008682c24d9 admin admin] PATH_INFO: `/v3/projects/5ff29c183342477c81bf4b8ed16faaa9/users/e970ec048057461b997e0900210cda0c/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.871565 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1251/2491] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:47.872719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d930188-66f7-4be2-9fa7-9bb6ec0e7294 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.883966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90477d0b-9c13-4832-bfe3-9008682c24d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff29c183342477c81bf4b8ed16faaa9, user_id=e970ec048057461b997e0900210cda0c, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.884967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d930188-66f7-4be2-9fa7-9bb6ec0e7294 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.888465 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90477d0b-9c13-4832-bfe3-9008682c24d9 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.889763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a359cfe-02a7-42ac-8e3e-9dfd5b2b218b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.891385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a359cfe-02a7-42ac-8e3e-9dfd5b2b218b admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.891607 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a359cfe-02a7-42ac-8e3e-9dfd5b2b218b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.891880 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a359cfe-02a7-42ac-8e3e-9dfd5b2b218b admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.892807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a359cfe-02a7-42ac-8e3e-9dfd5b2b218b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.894212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a359cfe-02a7-42ac-8e3e-9dfd5b2b218b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.905451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1252/2492] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:47.906307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a359cfe-02a7-42ac-8e3e-9dfd5b2b218b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.907171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1241/2493] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:47 2022] PUT /identity/v3/projects/5ff29c183342477c81bf4b8ed16faaa9/users/e970ec048057461b997e0900210cda0c/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:47.909553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90477d0b-9c13-4832-bfe3-9008682c24d9 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:47.921925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a359cfe-02a7-42ac-8e3e-9dfd5b2b218b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.985262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9af87-3428-46f5-8c42-f2287a39b296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.985961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9af87-3428-46f5-8c42-f2287a39b296 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.986239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9af87-3428-46f5-8c42-f2287a39b296 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.986520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9af87-3428-46f5-8c42-f2287a39b296 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:47.987257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9af87-3428-46f5-8c42-f2287a39b296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:47.987833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90477d0b-9c13-4832-bfe3-9008682c24d9 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:47.989160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9af87-3428-46f5-8c42-f2287a39b296 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:47.992715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7eaf82-41d7-47b1-b07f-b3db21433bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:47.993599 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7eaf82-41d7-47b1-b07f-b3db21433bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:47.993874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7eaf82-41d7-47b1-b07f-b3db21433bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:47.994142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7eaf82-41d7-47b1-b07f-b3db21433bdc admin admin] PATH_INFO: `/v3/projects/f3535011fbf74c7ebdcd5660b677084a/users/390fc417ac3e4685825ca61b6d820e90/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.013332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7eaf82-41d7-47b1-b07f-b3db21433bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3535011fbf74c7ebdcd5660b677084a, user_id=390fc417ac3e4685825ca61b6d820e90, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.017209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7eaf82-41d7-47b1-b07f-b3db21433bdc admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.026309 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1253/2494] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:47 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:48.027196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9af87-3428-46f5-8c42-f2287a39b296 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.038051 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1242/2495] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:47 2022] PUT /identity/v3/projects/f3535011fbf74c7ebdcd5660b677084a/users/390fc417ac3e4685825ca61b6d820e90/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:48.038290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7eaf82-41d7-47b1-b07f-b3db21433bdc admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.116600 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7eaf82-41d7-47b1-b07f-b3db21433bdc admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.119443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9af87-3428-46f5-8c42-f2287a39b296 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.122611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f16ed7-472a-4284-95e6-ee8ed440b1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.123472 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f16ed7-472a-4284-95e6-ee8ed440b1fd admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.123695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f16ed7-472a-4284-95e6-ee8ed440b1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.123994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f16ed7-472a-4284-95e6-ee8ed440b1fd admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.125446 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f16ed7-472a-4284-95e6-ee8ed440b1fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.127252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ccb3a-a3eb-4ea6-9cda-394f68bf9aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.127521 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f16ed7-472a-4284-95e6-ee8ed440b1fd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.129153 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ccb3a-a3eb-4ea6-9cda-394f68bf9aad admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.129698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ccb3a-a3eb-4ea6-9cda-394f68bf9aad admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.130295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ccb3a-a3eb-4ea6-9cda-394f68bf9aad admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.132007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ccb3a-a3eb-4ea6-9cda-394f68bf9aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.135263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ccb3a-a3eb-4ea6-9cda-394f68bf9aad admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.148307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1243/2496] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:48 2022] POST /identity/v3/users => generated 298 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:48.148307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42608c-706a-434b-9d22-e354062ca343 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.148307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42608c-706a-434b-9d22-e354062ca343 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.148307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42608c-706a-434b-9d22-e354062ca343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.153080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1254/2497] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:48.154284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ccb3a-a3eb-4ea6-9cda-394f68bf9aad admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.166336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ccb3a-a3eb-4ea6-9cda-394f68bf9aad admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.171719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ead59e-ef77-4162-994e-ef54ee90b440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.172538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ead59e-ef77-4162-994e-ef54ee90b440 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.172824 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ead59e-ef77-4162-994e-ef54ee90b440 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.173111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ead59e-ef77-4162-994e-ef54ee90b440 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.174105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ead59e-ef77-4162-994e-ef54ee90b440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.175918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ead59e-ef77-4162-994e-ef54ee90b440 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.181564 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-de42608c-706a-434b-9d22-e354062ca343 None None] MFA Rules not processed for user `390fc417ac3e4685825ca61b6d820e90`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:48.182953 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-de42608c-706a-434b-9d22-e354062ca343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:48.189366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1255/2498] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:48.190186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ead59e-ef77-4162-994e-ef54ee90b440 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.201129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ead59e-ef77-4162-994e-ef54ee90b440 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.207129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182bf3d-7cdd-4c7e-8fa0-47de6e6fdb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.208157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182bf3d-7cdd-4c7e-8fa0-47de6e6fdb92 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.208511 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182bf3d-7cdd-4c7e-8fa0-47de6e6fdb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.208890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182bf3d-7cdd-4c7e-8fa0-47de6e6fdb92 admin admin] PATH_INFO: `/v3/projects/5ff29c183342477c81bf4b8ed16faaa9/users/e970ec048057461b997e0900210cda0c/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.211093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182bf3d-7cdd-4c7e-8fa0-47de6e6fdb92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff29c183342477c81bf4b8ed16faaa9, user_id=e970ec048057461b997e0900210cda0c, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.213895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182bf3d-7cdd-4c7e-8fa0-47de6e6fdb92 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.232129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1256/2499] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:48 2022] PUT /identity/v3/projects/5ff29c183342477c81bf4b8ed16faaa9/users/e970ec048057461b997e0900210cda0c/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:48.232129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182bf3d-7cdd-4c7e-8fa0-47de6e6fdb92 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.255901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1244/2500] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:48 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:48.258970 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c3bd2-b9c9-429d-b03d-2cd9ff303465 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.259261 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c3bd2-b9c9-429d-b03d-2cd9ff303465 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.259536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c3bd2-b9c9-429d-b03d-2cd9ff303465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.289505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-626c3bd2-b9c9-429d-b03d-2cd9ff303465 None None] MFA Rules not processed for user `e970ec048057461b997e0900210cda0c`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:48.290728 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-626c3bd2-b9c9-429d-b03d-2cd9ff303465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:48.306141 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182bf3d-7cdd-4c7e-8fa0-47de6e6fdb92 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.310949 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606c8e9-3d9a-4a18-9cc9-cb9779048506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.312434 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606c8e9-3d9a-4a18-9cc9-cb9779048506 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.312717 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606c8e9-3d9a-4a18-9cc9-cb9779048506 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.312995 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606c8e9-3d9a-4a18-9cc9-cb9779048506 admin admin] PATH_INFO: `/v3/projects/5170a6f23fb54d51b621c9c5ebbbff96/users/f5ac350a626f4058b44c2cd704f84ba4/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.331255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606c8e9-3d9a-4a18-9cc9-cb9779048506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5170a6f23fb54d51b621c9c5ebbbff96, user_id=f5ac350a626f4058b44c2cd704f84ba4, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.333933 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606c8e9-3d9a-4a18-9cc9-cb9779048506 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.348217 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1257/2501] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:48 2022] PUT /identity/v3/projects/5170a6f23fb54d51b621c9c5ebbbff96/users/f5ac350a626f4058b44c2cd704f84ba4/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:48.351216 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606c8e9-3d9a-4a18-9cc9-cb9779048506 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.363702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1245/2502] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:48.368594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c3bd2-b9c9-429d-b03d-2cd9ff303465 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.381149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c3bd2-b9c9-429d-b03d-2cd9ff303465 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.413322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606c8e9-3d9a-4a18-9cc9-cb9779048506 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.418164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52a92f-5dfc-4cb2-beaa-6db598c17ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.418860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52a92f-5dfc-4cb2-beaa-6db598c17ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.419150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52a92f-5dfc-4cb2-beaa-6db598c17ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.419417 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52a92f-5dfc-4cb2-beaa-6db598c17ad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.420132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52a92f-5dfc-4cb2-beaa-6db598c17ad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.422870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b52a92f-5dfc-4cb2-beaa-6db598c17ad9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.439717 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879153e5-9452-49e0-a588-7297ee06f407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.440443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879153e5-9452-49e0-a588-7297ee06f407 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.440770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879153e5-9452-49e0-a588-7297ee06f407 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.441048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879153e5-9452-49e0-a588-7297ee06f407 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.441898 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879153e5-9452-49e0-a588-7297ee06f407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.443758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879153e5-9452-49e0-a588-7297ee06f407 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.450315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1258/2503] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:48 2022] POST /identity/v3/projects => generated 355 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:48.451370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52a92f-5dfc-4cb2-beaa-6db598c17ad9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.453375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9b52a92f-5dfc-4cb2-beaa-6db598c17ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:48.455430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1246/2504] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:48.456606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879153e5-9452-49e0-a588-7297ee06f407 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.467950 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879153e5-9452-49e0-a588-7297ee06f407 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.514243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52a92f-5dfc-4cb2-beaa-6db598c17ad9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.518129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7867e-f281-4a98-8860-d2b858ef0adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.518853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7867e-f281-4a98-8860-d2b858ef0adb service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.519143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7867e-f281-4a98-8860-d2b858ef0adb service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.519420 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7867e-f281-4a98-8860-d2b858ef0adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.526262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e0c8f-5cec-4535-8396-4e3422cc0920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.526974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e0c8f-5cec-4535-8396-4e3422cc0920 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.527355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e0c8f-5cec-4535-8396-4e3422cc0920 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.527622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e0c8f-5cec-4535-8396-4e3422cc0920 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.528203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e0c8f-5cec-4535-8396-4e3422cc0920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.530288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e0c8f-5cec-4535-8396-4e3422cc0920 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.554509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1247/2505] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:48 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:48.554509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e0c8f-5cec-4535-8396-4e3422cc0920 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.564631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e0c8f-5cec-4535-8396-4e3422cc0920 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.569533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a0e68-07c1-48a1-bbc2-6536166dd03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.570371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a0e68-07c1-48a1-bbc2-6536166dd03f admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.570657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a0e68-07c1-48a1-bbc2-6536166dd03f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.570997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a0e68-07c1-48a1-bbc2-6536166dd03f admin admin] PATH_INFO: `/v3/projects/5170a6f23fb54d51b621c9c5ebbbff96/users/f5ac350a626f4058b44c2cd704f84ba4/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.573141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a0e68-07c1-48a1-bbc2-6536166dd03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5170a6f23fb54d51b621c9c5ebbbff96, user_id=f5ac350a626f4058b44c2cd704f84ba4, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.576043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a0e68-07c1-48a1-bbc2-6536166dd03f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.579181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7867e-f281-4a98-8860-d2b858ef0adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.580629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7867e-f281-4a98-8860-d2b858ef0adb service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.587422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1248/2506] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:48 2022] PUT /identity/v3/projects/5170a6f23fb54d51b621c9c5ebbbff96/users/f5ac350a626f4058b44c2cd704f84ba4/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:48.588026 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a0e68-07c1-48a1-bbc2-6536166dd03f admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.590031 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1259/2507] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:48.599877 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7976e09-c11c-43c3-8ffc-1eb6d8a80915 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.600164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7976e09-c11c-43c3-8ffc-1eb6d8a80915 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.600442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7976e09-c11c-43c3-8ffc-1eb6d8a80915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.630081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-d7976e09-c11c-43c3-8ffc-1eb6d8a80915 None None] MFA Rules not processed for user `f5ac350a626f4058b44c2cd704f84ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:48.631431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d7976e09-c11c-43c3-8ffc-1eb6d8a80915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:48.659027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a0e68-07c1-48a1-bbc2-6536166dd03f admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.667146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500c547-37e4-45ce-9048-a36be351e870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.668441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500c547-37e4-45ce-9048-a36be351e870 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.669146 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500c547-37e4-45ce-9048-a36be351e870 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.669784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500c547-37e4-45ce-9048-a36be351e870 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.670304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500c547-37e4-45ce-9048-a36be351e870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.673654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500c547-37e4-45ce-9048-a36be351e870 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.691384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1249/2508] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:48.695628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500c547-37e4-45ce-9048-a36be351e870 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.706518 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500c547-37e4-45ce-9048-a36be351e870 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.711945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7935eb-c439-4e8e-90eb-8fddd7500a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.712323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1260/2509] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:48.712828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7935eb-c439-4e8e-90eb-8fddd7500a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.713106 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7935eb-c439-4e8e-90eb-8fddd7500a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.713372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7935eb-c439-4e8e-90eb-8fddd7500a06 admin admin] PATH_INFO: `/v3/projects/411823cad8e14c259bfdd9c0816e2e47/users/db82d57a3af447f2a3045555a3c73029/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.724895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7976e09-c11c-43c3-8ffc-1eb6d8a80915 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.726416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d7976e09-c11c-43c3-8ffc-1eb6d8a80915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:48.740583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7935eb-c439-4e8e-90eb-8fddd7500a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=411823cad8e14c259bfdd9c0816e2e47, user_id=db82d57a3af447f2a3045555a3c73029, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.744655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7935eb-c439-4e8e-90eb-8fddd7500a06 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.767060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1250/2510] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:48 2022] PUT /identity/v3/projects/411823cad8e14c259bfdd9c0816e2e47/users/db82d57a3af447f2a3045555a3c73029/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:48.768167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7935eb-c439-4e8e-90eb-8fddd7500a06 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.805668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7976e09-c11c-43c3-8ffc-1eb6d8a80915 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.857951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7935eb-c439-4e8e-90eb-8fddd7500a06 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.861361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe593ec-d253-4aa8-9262-ed463174530f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.861912 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe593ec-d253-4aa8-9262-ed463174530f service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.862200 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe593ec-d253-4aa8-9262-ed463174530f service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.862481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe593ec-d253-4aa8-9262-ed463174530f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.863628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba2004-23bd-47de-8596-500c726c6836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.864398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba2004-23bd-47de-8596-500c726c6836 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.864755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba2004-23bd-47de-8596-500c726c6836 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.865087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba2004-23bd-47de-8596-500c726c6836 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.865877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba2004-23bd-47de-8596-500c726c6836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.867584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba2004-23bd-47de-8596-500c726c6836 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.879475 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1251/2511] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:48.883559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba2004-23bd-47de-8596-500c726c6836 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:48.896061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba2004-23bd-47de-8596-500c726c6836 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:48.903636 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34ec42-3ed0-4cba-a32e-6f01c97a94db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:48.904681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34ec42-3ed0-4cba-a32e-6f01c97a94db admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.904976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34ec42-3ed0-4cba-a32e-6f01c97a94db admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.905244 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34ec42-3ed0-4cba-a32e-6f01c97a94db admin admin] PATH_INFO: `/v3/projects/411823cad8e14c259bfdd9c0816e2e47/users/db82d57a3af447f2a3045555a3c73029/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.908041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34ec42-3ed0-4cba-a32e-6f01c97a94db admin admin] RBAC: Authorizing `identity:create_grant(project_id=411823cad8e14c259bfdd9c0816e2e47, user_id=db82d57a3af447f2a3045555a3c73029, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.912510 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34ec42-3ed0-4cba-a32e-6f01c97a94db admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.924488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1252/2512] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:48 2022] PUT /identity/v3/projects/411823cad8e14c259bfdd9c0816e2e47/users/db82d57a3af447f2a3045555a3c73029/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:48.936316 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8a23-9d94-45c0-84b6-bbb1a4912941 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:48.937293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8a23-9d94-45c0-84b6-bbb1a4912941 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:48.937674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8a23-9d94-45c0-84b6-bbb1a4912941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:48.948857 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe593ec-d253-4aa8-9262-ed463174530f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:48.950478 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe593ec-d253-4aa8-9262-ed463174530f service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:48.961602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1261/2513] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:48.970673 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-baaa8a23-9d94-45c0-84b6-bbb1a4912941 None None] MFA Rules not processed for user `db82d57a3af447f2a3045555a3c73029`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:48.972575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-baaa8a23-9d94-45c0-84b6-bbb1a4912941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:49.056828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1253/2514] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:48 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:49.070321 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe593ec-d253-4aa8-9262-ed463174530f service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:49.084079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe593ec-d253-4aa8-9262-ed463174530f service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:49.139221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ae01f-4711-4938-8abd-b6278970435e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:49.139916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ae01f-4711-4938-8abd-b6278970435e service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:49.140197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ae01f-4711-4938-8abd-b6278970435e service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:49.140472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ae01f-4711-4938-8abd-b6278970435e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:49.149962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ae01f-4711-4938-8abd-b6278970435e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:49.151464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ae01f-4711-4938-8abd-b6278970435e service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:49.161895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1262/2515] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:49 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:49.249183 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa8a23-9d94-45c0-84b6-bbb1a4912941 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:49.259866 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa8a23-9d94-45c0-84b6-bbb1a4912941 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:49.263842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc5337-ea91-48f5-a23f-3afa66049d87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:49.264607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc5337-ea91-48f5-a23f-3afa66049d87 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:49.265187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc5337-ea91-48f5-a23f-3afa66049d87 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:49.265530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc5337-ea91-48f5-a23f-3afa66049d87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:49.332997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc5337-ea91-48f5-a23f-3afa66049d87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:49.334342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc5337-ea91-48f5-a23f-3afa66049d87 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:49.344287 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1254/2516] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:49 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:03:50.684719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ae01f-4711-4938-8abd-b6278970435e service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:50.694194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ae01f-4711-4938-8abd-b6278970435e service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:50.698409 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74936a2b-c1f5-4a4d-9dbc-660924139b45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:50.699251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74936a2b-c1f5-4a4d-9dbc-660924139b45 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:50.699537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74936a2b-c1f5-4a4d-9dbc-660924139b45 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:50.699803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74936a2b-c1f5-4a4d-9dbc-660924139b45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:50.767003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74936a2b-c1f5-4a4d-9dbc-660924139b45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:50.768531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74936a2b-c1f5-4a4d-9dbc-660924139b45 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:50.778559 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1263/2517] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:51.204748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc5337-ea91-48f5-a23f-3afa66049d87 service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:51.303576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc5337-ea91-48f5-a23f-3afa66049d87 service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:51.303576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:51.303576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:51.303576 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:51.303956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] PATH_INFO: `/v3/users/cc88176d94bc498588e1702e4c8b1370` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:51.303956 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc88176d94bc498588e1702e4c8b1370)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:51.304562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:51.367985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc88176d94bc498588e1702e4c8b1370'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:51.368958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc88176d94bc498588e1702e4c8b1370'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:51.371109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74936a2b-c1f5-4a4d-9dbc-660924139b45 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:51.381127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74936a2b-c1f5-4a4d-9dbc-660924139b45 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:51.384667 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc88176d94bc498588e1702e4c8b1370'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:51.384989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51071ac4-07f7-4e79-a869-81754a636bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:51.385416 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc88176d94bc498588e1702e4c8b1370'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:51.385637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51071ac4-07f7-4e79-a869-81754a636bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:51.385925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51071ac4-07f7-4e79-a869-81754a636bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:51.386198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51071ac4-07f7-4e79-a869-81754a636bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:51.388016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-51071ac4-07f7-4e79-a869-81754a636bf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:51.392841 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0cb2af6e-bf18-41c7-a341-2eda734a935c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc88176d94bc498588e1702e4c8b1370'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:51.407373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1255/2518] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:51 2022] DELETE /identity/v3/users/cc88176d94bc498588e1702e4c8b1370 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:51.411892 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc0037-263a-4a21-8cdf-72f04964c570 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:51.412252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc0037-263a-4a21-8cdf-72f04964c570 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:51.412604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc0037-263a-4a21-8cdf-72f04964c570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:51.434543 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-26fc0037-263a-4a21-8cdf-72f04964c570 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:51.436469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-26fc0037-263a-4a21-8cdf-72f04964c570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:51.458931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51071ac4-07f7-4e79-a869-81754a636bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:51.460381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51071ac4-07f7-4e79-a869-81754a636bf9 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:51.470439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1264/2519] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:03:51.502823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1256/2520] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:51.513402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51071ac4-07f7-4e79-a869-81754a636bf9 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:51.515313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-51071ac4-07f7-4e79-a869-81754a636bf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:51.598101 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51071ac4-07f7-4e79-a869-81754a636bf9 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:51.648439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3dad1-0ba1-45c2-b5e9-1fef84cf4bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:51.649087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3dad1-0ba1-45c2-b5e9-1fef84cf4bcf service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:51.649368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3dad1-0ba1-45c2-b5e9-1fef84cf4bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:51.649641 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3dad1-0ba1-45c2-b5e9-1fef84cf4bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:51.659405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3dad1-0ba1-45c2-b5e9-1fef84cf4bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:51.660841 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3dad1-0ba1-45c2-b5e9-1fef84cf4bcf service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:51.670568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1265/2521] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:51.971125 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc0037-263a-4a21-8cdf-72f04964c570 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:51.977873 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-26fc0037-263a-4a21-8cdf-72f04964c570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:51.992964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc0037-263a-4a21-8cdf-72f04964c570 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:52.003997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:52.004941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:52.005234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:52.005540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] PATH_INFO: `/v3/projects/93dfe640b38f45f79fd2315cd7463f6b` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:52.006820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93dfe640b38f45f79fd2315cd7463f6b)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:52.008765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:52.066685 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93dfe640b38f45f79fd2315cd7463f6b', 'request_id': 'req-42acea98-a385-403c-a045-658f68f20db1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.076826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93dfe640b38f45f79fd2315cd7463f6b', 'request_id': 'req-42acea98-a385-403c-a045-658f68f20db1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.089620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93dfe640b38f45f79fd2315cd7463f6b', 'request_id': 'req-42acea98-a385-403c-a045-658f68f20db1'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.107063 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] The token cache is being invalidate because project 93dfe640b38f45f79fd2315cd7463f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:52.107350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.108145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.110838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1257/2522] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:51 2022] DELETE /identity/v3/projects/93dfe640b38f45f79fd2315cd7463f6b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:52.117172 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3dad1-0ba1-45c2-b5e9-1fef84cf4bcf service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:52.129805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3dad1-0ba1-45c2-b5e9-1fef84cf4bcf service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:52.213477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:52.214348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:52.214627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:52.214944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] PATH_INFO: `/v3/users/e970ec048057461b997e0900210cda0c` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:52.216132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e970ec048057461b997e0900210cda0c)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:52.218609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:52.267753 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e970ec048057461b997e0900210cda0c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.282524 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e970ec048057461b997e0900210cda0c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.289968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e970ec048057461b997e0900210cda0c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.290689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e970ec048057461b997e0900210cda0c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.299408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21160058-4784-40e2-8097-10dc7145a67f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e970ec048057461b997e0900210cda0c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.306422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1266/2523] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:52 2022] DELETE /identity/v3/users/e970ec048057461b997e0900210cda0c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:03:52.595102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:52.596771 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:52.614528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acea98-a385-403c-a045-658f68f20db1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:52.683427 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:52.684386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:52.684671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:52.684966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] PATH_INFO: `/v3/projects/5ff29c183342477c81bf4b8ed16faaa9` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:52.686211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff29c183342477c81bf4b8ed16faaa9)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:52.688203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:52.744908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff29c183342477c81bf4b8ed16faaa9', 'request_id': 'req-524c41bd-7882-4a46-a959-fff84a76b58b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.751450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff29c183342477c81bf4b8ed16faaa9', 'request_id': 'req-524c41bd-7882-4a46-a959-fff84a76b58b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.757598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff29c183342477c81bf4b8ed16faaa9', 'request_id': 'req-524c41bd-7882-4a46-a959-fff84a76b58b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.772410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] The token cache is being invalidate because project 5ff29c183342477c81bf4b8ed16faaa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:52.772908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.773739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-524c41bd-7882-4a46-a959-fff84a76b58b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:52.775360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1258/2524] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:52 2022] DELETE /identity/v3/projects/5ff29c183342477c81bf4b8ed16faaa9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:52.784895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe925-d218-48f4-85dc-f9b90e33271d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:52.785839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe925-d218-48f4-85dc-f9b90e33271d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:52.786166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe925-d218-48f4-85dc-f9b90e33271d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:52.805183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-c47fe925-d218-48f4-85dc-f9b90e33271d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:52.806698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c47fe925-d218-48f4-85dc-f9b90e33271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:52.895361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1267/2525] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:52.907947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f2530-e827-4e9d-af09-6983f31ca732 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:52.913384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f2530-e827-4e9d-af09-6983f31ca732 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:52.913384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f2530-e827-4e9d-af09-6983f31ca732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:52.933205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-f19f2530-e827-4e9d-af09-6983f31ca732 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:52.934785 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f19f2530-e827-4e9d-af09-6983f31ca732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:52.948706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1259/2526] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:52.953655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fe925-d218-48f4-85dc-f9b90e33271d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:52.963121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fe925-d218-48f4-85dc-f9b90e33271d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:52.968644 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583607cb-f2d2-4e29-8a55-cd12999eb104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:52.969232 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583607cb-f2d2-4e29-8a55-cd12999eb104 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:52.969508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583607cb-f2d2-4e29-8a55-cd12999eb104 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:52.969773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583607cb-f2d2-4e29-8a55-cd12999eb104 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:52.970506 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583607cb-f2d2-4e29-8a55-cd12999eb104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:52.971269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583607cb-f2d2-4e29-8a55-cd12999eb104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:52.972876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583607cb-f2d2-4e29-8a55-cd12999eb104 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:52.986261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1268/2527] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:03:52.991242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f2530-e827-4e9d-af09-6983f31ca732 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:53.001433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f2530-e827-4e9d-af09-6983f31ca732 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:53.006475 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92aedf-67eb-4913-b645-59405dcb97b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:53.007138 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92aedf-67eb-4913-b645-59405dcb97b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:53.007560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92aedf-67eb-4913-b645-59405dcb97b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:53.007826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92aedf-67eb-4913-b645-59405dcb97b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:53.008519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92aedf-67eb-4913-b645-59405dcb97b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:53.010398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92aedf-67eb-4913-b645-59405dcb97b5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:53.038226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1260/2528] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:52 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:53.042470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583607cb-f2d2-4e29-8a55-cd12999eb104 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:53.053802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583607cb-f2d2-4e29-8a55-cd12999eb104 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:53.123488 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94700048-1306-4a15-bc2c-e9225e6240b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:53.125144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94700048-1306-4a15-bc2c-e9225e6240b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:53.125637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94700048-1306-4a15-bc2c-e9225e6240b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:53.125967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94700048-1306-4a15-bc2c-e9225e6240b8 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:53.126822 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94700048-1306-4a15-bc2c-e9225e6240b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:53.128918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94700048-1306-4a15-bc2c-e9225e6240b8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:53.151529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1269/2529] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:53 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:03:53.156816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92aedf-67eb-4913-b645-59405dcb97b5 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:53.172246 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92aedf-67eb-4913-b645-59405dcb97b5 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:53.177454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f057e5-be96-4ee7-a525-f5116a9af278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:53.178256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f057e5-be96-4ee7-a525-f5116a9af278 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:53.178536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f057e5-be96-4ee7-a525-f5116a9af278 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:53.178895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f057e5-be96-4ee7-a525-f5116a9af278 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:53.179765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f057e5-be96-4ee7-a525-f5116a9af278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:53.181847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f057e5-be96-4ee7-a525-f5116a9af278 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:53.194181 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1261/2530] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:53.210527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94700048-1306-4a15-bc2c-e9225e6240b8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:53.225038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94700048-1306-4a15-bc2c-e9225e6240b8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:53.230308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2566c-2dda-4192-afbb-3c4180b8d01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:53.231242 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2566c-2dda-4192-afbb-3c4180b8d01d admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:53.231637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2566c-2dda-4192-afbb-3c4180b8d01d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:53.231984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2566c-2dda-4192-afbb-3c4180b8d01d admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:53.233646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2566c-2dda-4192-afbb-3c4180b8d01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:53.235647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2566c-2dda-4192-afbb-3c4180b8d01d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:53.255350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1270/2531] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:53.262333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f057e5-be96-4ee7-a525-f5116a9af278 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:53.280363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f057e5-be96-4ee7-a525-f5116a9af278 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:53.288697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c5ac0-35d4-41d2-a684-d9e3a0d71f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:53.290118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c5ac0-35d4-41d2-a684-d9e3a0d71f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:53.290571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c5ac0-35d4-41d2-a684-d9e3a0d71f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:53.291062 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c5ac0-35d4-41d2-a684-d9e3a0d71f30 admin admin] PATH_INFO: `/v3/projects/484e74418bcc4b398bad5b2de6ff220c/users/cd58b86b905e42aeb529904b0270636c/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:53.315297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c5ac0-35d4-41d2-a684-d9e3a0d71f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484e74418bcc4b398bad5b2de6ff220c, user_id=cd58b86b905e42aeb529904b0270636c, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:53.318189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c5ac0-35d4-41d2-a684-d9e3a0d71f30 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:53.337090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1262/2532] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:53 2022] PUT /identity/v3/projects/484e74418bcc4b398bad5b2de6ff220c/users/cd58b86b905e42aeb529904b0270636c/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:53.424749 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4c18d-417b-4289-a1fb-0acf80d8ee4d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:53.425951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4c18d-417b-4289-a1fb-0acf80d8ee4d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:53.425951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4c18d-417b-4289-a1fb-0acf80d8ee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:53.455843 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-2ec4c18d-417b-4289-a1fb-0acf80d8ee4d None None] MFA Rules not processed for user `cd58b86b905e42aeb529904b0270636c`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:53.457089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2ec4c18d-417b-4289-a1fb-0acf80d8ee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:53.568314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1271/2533] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:03:53.581319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c5ac0-35d4-41d2-a684-d9e3a0d71f30 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:53.583353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-693c5ac0-35d4-41d2-a684-d9e3a0d71f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:53.655394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c5ac0-35d4-41d2-a684-d9e3a0d71f30 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:53.660781 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d073733-e5af-47b0-9eef-fb2a91b535c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:53.661432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d073733-e5af-47b0-9eef-fb2a91b535c1 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:53.661710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d073733-e5af-47b0-9eef-fb2a91b535c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:53.661983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d073733-e5af-47b0-9eef-fb2a91b535c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:53.672090 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d073733-e5af-47b0-9eef-fb2a91b535c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:53.673546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d073733-e5af-47b0-9eef-fb2a91b535c1 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:53.684069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1263/2534] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:55.503356 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4c18d-417b-4289-a1fb-0acf80d8ee4d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:55.504798 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2ec4c18d-417b-4289-a1fb-0acf80d8ee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:55.578977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4c18d-417b-4289-a1fb-0acf80d8ee4d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:55.583884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:55.584761 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:55.585064 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:55.585331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] PATH_INFO: `/v3/users/2f9db02e76b2432d8cef24abd2570c83` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:55.586545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9db02e76b2432d8cef24abd2570c83)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:55.588500 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:55.640839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9db02e76b2432d8cef24abd2570c83'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:55.652491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9db02e76b2432d8cef24abd2570c83'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:55.659887 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9db02e76b2432d8cef24abd2570c83'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:55.661405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9db02e76b2432d8cef24abd2570c83'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:55.669132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9db02e76b2432d8cef24abd2570c83'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:55.682055 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1272/2535] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:55 2022] DELETE /identity/v3/users/2f9db02e76b2432d8cef24abd2570c83 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:55.688813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f390d-20df-468e-9aa4-33d695369945 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:55.689328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f390d-20df-468e-9aa4-33d695369945 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:55.689614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f390d-20df-468e-9aa4-33d695369945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:55.713528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-522f390d-20df-468e-9aa4-33d695369945 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:55.715272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-522f390d-20df-468e-9aa4-33d695369945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:55.792913 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1264/2536] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:55.803681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:55.805258 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:55.883063 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86e800-8b44-4a97-a809-a2d00f9317a1 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:55.888666 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057397a-0abd-41df-9b8a-20a69557740f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:55.889646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057397a-0abd-41df-9b8a-20a69557740f service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:55.890608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057397a-0abd-41df-9b8a-20a69557740f service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:55.890969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057397a-0abd-41df-9b8a-20a69557740f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:55.903288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6057397a-0abd-41df-9b8a-20a69557740f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:55.904823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6057397a-0abd-41df-9b8a-20a69557740f service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:55.921454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1273/2537] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:03:56.324343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f390d-20df-468e-9aa4-33d695369945 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:56.326150 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-522f390d-20df-468e-9aa4-33d695369945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:56.340744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f390d-20df-468e-9aa4-33d695369945 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:56.352477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:56.353986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:56.354493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:56.355325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] PATH_INFO: `/v3/projects/b85eeb0a7ee0408ca90bb2e4c23f0543` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:56.358104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b85eeb0a7ee0408ca90bb2e4c23f0543)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:56.363338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:56.434909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b85eeb0a7ee0408ca90bb2e4c23f0543', 'request_id': 'req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:56.445003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b85eeb0a7ee0408ca90bb2e4c23f0543', 'request_id': 'req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:56.452184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b85eeb0a7ee0408ca90bb2e4c23f0543', 'request_id': 'req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:56.466772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] The token cache is being invalidate because project b85eeb0a7ee0408ca90bb2e4c23f0543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:56.467067 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:56.467817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:56.469966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1265/2538] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:56 2022] DELETE /identity/v3/projects/b85eeb0a7ee0408ca90bb2e4c23f0543 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:56.474096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057397a-0abd-41df-9b8a-20a69557740f service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:56.484889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057397a-0abd-41df-9b8a-20a69557740f service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:56.562333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:56.563305 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:56.564519 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:56.564750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] PATH_INFO: `/v3/users/90491414026748558450f28759ea6d41` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:56.566477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90491414026748558450f28759ea6d41)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:56.568470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:56.632978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90491414026748558450f28759ea6d41'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:56.653604 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90491414026748558450f28759ea6d41'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:56.661847 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90491414026748558450f28759ea6d41'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:56.662590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90491414026748558450f28759ea6d41'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:56.670451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90491414026748558450f28759ea6d41'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:56.682515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1274/2539] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:56 2022] DELETE /identity/v3/users/90491414026748558450f28759ea6d41 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:56.973477 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:56.975045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:56.989354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7564ae-6f99-40fb-b77a-20e7ed9667bd admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:57.049979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:57.050791 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:57.051056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:57.051328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] PATH_INFO: `/v3/projects/b886bb95424c43b98f23a94de334b3fb` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:57.052528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b886bb95424c43b98f23a94de334b3fb)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:57.054431 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:57.103257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b886bb95424c43b98f23a94de334b3fb', 'request_id': 'req-b625d884-2ab1-456b-8cfe-8320cbbe3b44'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.109476 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b886bb95424c43b98f23a94de334b3fb', 'request_id': 'req-b625d884-2ab1-456b-8cfe-8320cbbe3b44'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.115133 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b886bb95424c43b98f23a94de334b3fb', 'request_id': 'req-b625d884-2ab1-456b-8cfe-8320cbbe3b44'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.130257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] The token cache is being invalidate because project b886bb95424c43b98f23a94de334b3fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:57.130747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.132144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.133992 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1266/2540] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:56 2022] DELETE /identity/v3/projects/b886bb95424c43b98f23a94de334b3fb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:57.138280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:57.139779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:57.153632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6b27c-9f4f-4281-aaff-c8ba04539566 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:57.212033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:57.212916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:57.213188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:57.213458 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] PATH_INFO: `/v3/users/f5ac350a626f4058b44c2cd704f84ba4` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:57.214959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ac350a626f4058b44c2cd704f84ba4)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:57.217327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:57.300093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ac350a626f4058b44c2cd704f84ba4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.314061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ac350a626f4058b44c2cd704f84ba4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.321357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ac350a626f4058b44c2cd704f84ba4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.322158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ac350a626f4058b44c2cd704f84ba4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.330145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-3a378b71-b611-4ece-83c0-2870cb6a14f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ac350a626f4058b44c2cd704f84ba4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.337000 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1275/2541] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:03:57 2022] DELETE /identity/v3/users/f5ac350a626f4058b44c2cd704f84ba4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:57.656544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:57.658749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:57.677494 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625d884-2ab1-456b-8cfe-8320cbbe3b44 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:57.732900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:57.734355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:57.734638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:57.734947 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] PATH_INFO: `/v3/projects/5170a6f23fb54d51b621c9c5ebbbff96` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:57.736207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5170a6f23fb54d51b621c9c5ebbbff96)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:57.738132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:57.789878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5170a6f23fb54d51b621c9c5ebbbff96', 'request_id': 'req-29aca306-04c2-44a2-9bf9-fd3ffb88490c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.798040 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5170a6f23fb54d51b621c9c5ebbbff96', 'request_id': 'req-29aca306-04c2-44a2-9bf9-fd3ffb88490c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.804523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5170a6f23fb54d51b621c9c5ebbbff96', 'request_id': 'req-29aca306-04c2-44a2-9bf9-fd3ffb88490c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.822739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] The token cache is being invalidate because project 5170a6f23fb54d51b621c9c5ebbbff96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:03:57.823020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.823654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-29aca306-04c2-44a2-9bf9-fd3ffb88490c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:03:57.825180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1267/2542] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:03:57 2022] DELETE /identity/v3/projects/5170a6f23fb54d51b621c9c5ebbbff96 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:57.830621 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f41efc-41b9-442f-a0bb-fffbac2dfe01 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:57.831199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f41efc-41b9-442f-a0bb-fffbac2dfe01 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:57.831199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f41efc-41b9-442f-a0bb-fffbac2dfe01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:57.850544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f7f41efc-41b9-442f-a0bb-fffbac2dfe01 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:57.851772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f7f41efc-41b9-442f-a0bb-fffbac2dfe01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:57.920805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1276/2543] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:57.933998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136049f2-50f3-4d3d-a4fa-e162b6b28504 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:57.934299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136049f2-50f3-4d3d-a4fa-e162b6b28504 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:57.934570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136049f2-50f3-4d3d-a4fa-e162b6b28504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:57.968379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-136049f2-50f3-4d3d-a4fa-e162b6b28504 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:57.969891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-136049f2-50f3-4d3d-a4fa-e162b6b28504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:57.991914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1268/2544] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:57.996651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f41efc-41b9-442f-a0bb-fffbac2dfe01 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:58.009030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f41efc-41b9-442f-a0bb-fffbac2dfe01 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:58.013830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c995af9-c939-4af2-9a95-b75bc5ae26a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:58.014491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c995af9-c939-4af2-9a95-b75bc5ae26a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:58.014800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c995af9-c939-4af2-9a95-b75bc5ae26a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:58.015136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c995af9-c939-4af2-9a95-b75bc5ae26a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:58.015863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c995af9-c939-4af2-9a95-b75bc5ae26a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:03:58.016166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c995af9-c939-4af2-9a95-b75bc5ae26a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:58.018092 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c995af9-c939-4af2-9a95-b75bc5ae26a0 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:58.040853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1277/2545] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:03:58.046972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136049f2-50f3-4d3d-a4fa-e162b6b28504 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:58.057875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136049f2-50f3-4d3d-a4fa-e162b6b28504 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:58.071832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9b297-5520-4987-8957-52a95d67af00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:58.071832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9b297-5520-4987-8957-52a95d67af00 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:58.071832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9b297-5520-4987-8957-52a95d67af00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:58.071832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9b297-5520-4987-8957-52a95d67af00 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:58.072368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9b297-5520-4987-8957-52a95d67af00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:58.072368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9b297-5520-4987-8957-52a95d67af00 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:58.096128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1269/2546] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:03:58 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:58.100425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c995af9-c939-4af2-9a95-b75bc5ae26a0 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:58.112185 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c995af9-c939-4af2-9a95-b75bc5ae26a0 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:58.182978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f24d4-53cf-417c-aa35-74b7c2cba3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:58.183655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f24d4-53cf-417c-aa35-74b7c2cba3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:58.183935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f24d4-53cf-417c-aa35-74b7c2cba3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:58.184208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f24d4-53cf-417c-aa35-74b7c2cba3b6 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:58.184908 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f24d4-53cf-417c-aa35-74b7c2cba3b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:58.186914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f24d4-53cf-417c-aa35-74b7c2cba3b6 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:58.209148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1278/2547] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:03:58 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:03:58.213298 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9b297-5520-4987-8957-52a95d67af00 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:58.224417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9b297-5520-4987-8957-52a95d67af00 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:58.229379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1988097-08f1-4732-89a3-4e36cd542770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:58.230077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1988097-08f1-4732-89a3-4e36cd542770 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:58.230423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1988097-08f1-4732-89a3-4e36cd542770 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:58.230619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1988097-08f1-4732-89a3-4e36cd542770 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:58.232115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1988097-08f1-4732-89a3-4e36cd542770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:58.233038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1988097-08f1-4732-89a3-4e36cd542770 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:58.244256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1270/2548] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:58.248513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f24d4-53cf-417c-aa35-74b7c2cba3b6 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:58.258914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f24d4-53cf-417c-aa35-74b7c2cba3b6 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:58.264067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be728bab-93f2-4482-b54d-11932377d926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:58.264739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be728bab-93f2-4482-b54d-11932377d926 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:58.265529 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be728bab-93f2-4482-b54d-11932377d926 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:58.265812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be728bab-93f2-4482-b54d-11932377d926 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:58.266538 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be728bab-93f2-4482-b54d-11932377d926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:58.268177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be728bab-93f2-4482-b54d-11932377d926 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:58.280012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1279/2549] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:03:58.286742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1988097-08f1-4732-89a3-4e36cd542770 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:58.296230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1988097-08f1-4732-89a3-4e36cd542770 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:58.300993 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333c870-8296-4e4e-89c2-efb26a7b45d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:58.301916 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333c870-8296-4e4e-89c2-efb26a7b45d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:58.302196 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333c870-8296-4e4e-89c2-efb26a7b45d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:58.302473 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333c870-8296-4e4e-89c2-efb26a7b45d2 admin admin] PATH_INFO: `/v3/projects/37646edfd30f4131b251f5c649b423bc/users/7862bb758fc7492f866c1cae8d539eee/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:58.326879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333c870-8296-4e4e-89c2-efb26a7b45d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37646edfd30f4131b251f5c649b423bc, user_id=7862bb758fc7492f866c1cae8d539eee, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:58.329836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333c870-8296-4e4e-89c2-efb26a7b45d2 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:58.348292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1271/2550] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:03:58 2022] PUT /identity/v3/projects/37646edfd30f4131b251f5c649b423bc/users/7862bb758fc7492f866c1cae8d539eee/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:03:58.359338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4f78-ba2f-4932-be96-2bb8d5e065fa None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:58.359617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4f78-ba2f-4932-be96-2bb8d5e065fa None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:58.359891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4f78-ba2f-4932-be96-2bb8d5e065fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:58.389043 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-360e4f78-ba2f-4932-be96-2bb8d5e065fa None None] MFA Rules not processed for user `7862bb758fc7492f866c1cae8d539eee`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:03:58.390881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-360e4f78-ba2f-4932-be96-2bb8d5e065fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:58.478526 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1280/2551] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:03:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:03:58.491941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333c870-8296-4e4e-89c2-efb26a7b45d2 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:03:58.494432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a333c870-8296-4e4e-89c2-efb26a7b45d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:03:58.587859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333c870-8296-4e4e-89c2-efb26a7b45d2 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:03:58.594603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42f948-63b5-454d-ba1b-eb928ec0a931 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4f45e859d84ceaa5a01477e83e098e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:03:58.595388 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42f948-63b5-454d-ba1b-eb928ec0a931 service glance] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:03:58.595775 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42f948-63b5-454d-ba1b-eb928ec0a931 service glance] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:03:58.596084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42f948-63b5-454d-ba1b-eb928ec0a931 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:03:58.609645 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42f948-63b5-454d-ba1b-eb928ec0a931 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:03:58.612344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42f948-63b5-454d-ba1b-eb928ec0a931 service glance] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:03:58.627064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1272/2552] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:03:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:08.601294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4f78-ba2f-4932-be96-2bb8d5e065fa None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:08.603615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-360e4f78-ba2f-4932-be96-2bb8d5e065fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:08.720073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4f78-ba2f-4932-be96-2bb8d5e065fa None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:08.727570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:08.729085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:08.729570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:08.729982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] PATH_INFO: `/v3/users/390fc417ac3e4685825ca61b6d820e90` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:08.732215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=390fc417ac3e4685825ca61b6d820e90)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:08.736700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:08.809628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '390fc417ac3e4685825ca61b6d820e90'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:08.827952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '390fc417ac3e4685825ca61b6d820e90'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:08.838275 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '390fc417ac3e4685825ca61b6d820e90'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:08.839610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '390fc417ac3e4685825ca61b6d820e90'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:08.850979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '390fc417ac3e4685825ca61b6d820e90'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:08.860624 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1281/2553] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:08 2022] DELETE /identity/v3/users/390fc417ac3e4685825ca61b6d820e90 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:08.865221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f024403-2817-409b-a451-b5894d6b84b5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:08.866113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f024403-2817-409b-a451-b5894d6b84b5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:08.866675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f024403-2817-409b-a451-b5894d6b84b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:08.890510 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-8f024403-2817-409b-a451-b5894d6b84b5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:08.891800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8f024403-2817-409b-a451-b5894d6b84b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:08.982080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1273/2554] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:08.999845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:09.001750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:09.086106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e4996-77f8-4489-baf4-f7e91f057c1b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:09.094144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87bad2-070e-4e81-bb55-afe58a6269e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:09.094851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87bad2-070e-4e81-bb55-afe58a6269e5 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:09.095127 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87bad2-070e-4e81-bb55-afe58a6269e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:09.095395 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87bad2-070e-4e81-bb55-afe58a6269e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:09.111262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87bad2-070e-4e81-bb55-afe58a6269e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:09.112751 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87bad2-070e-4e81-bb55-afe58a6269e5 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:09.126584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1282/2555] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:09.504939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f024403-2817-409b-a451-b5894d6b84b5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:09.506580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8f024403-2817-409b-a451-b5894d6b84b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:09.521985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f024403-2817-409b-a451-b5894d6b84b5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:09.528130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:09.529155 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:09.529434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:09.529972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] PATH_INFO: `/v3/projects/f3535011fbf74c7ebdcd5660b677084a` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:09.536895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3535011fbf74c7ebdcd5660b677084a)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:09.539222 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:09.599975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3535011fbf74c7ebdcd5660b677084a', 'request_id': 'req-4f73e492-6bc1-47a8-b7a5-40d42da749d6'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:09.611844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3535011fbf74c7ebdcd5660b677084a', 'request_id': 'req-4f73e492-6bc1-47a8-b7a5-40d42da749d6'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:09.622481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3535011fbf74c7ebdcd5660b677084a', 'request_id': 'req-4f73e492-6bc1-47a8-b7a5-40d42da749d6'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:09.642974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] The token cache is being invalidate because project f3535011fbf74c7ebdcd5660b677084a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:09.643307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:09.644412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:09.646946 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1274/2556] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:09 2022] DELETE /identity/v3/projects/f3535011fbf74c7ebdcd5660b677084a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:09.652558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87bad2-070e-4e81-bb55-afe58a6269e5 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:09.666759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87bad2-070e-4e81-bb55-afe58a6269e5 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:09.749627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:09.751160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:09.751709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:09.752090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] PATH_INFO: `/v3/users/db82d57a3af447f2a3045555a3c73029` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:09.753932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db82d57a3af447f2a3045555a3c73029)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:09.756915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:09.823713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db82d57a3af447f2a3045555a3c73029'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:09.839628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db82d57a3af447f2a3045555a3c73029'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:09.852153 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db82d57a3af447f2a3045555a3c73029'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:09.853389 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db82d57a3af447f2a3045555a3c73029'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:09.863677 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-d7d5a559-41dd-4e3b-8f73-7b6aaafe0508 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db82d57a3af447f2a3045555a3c73029'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:09.875325 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1283/2557] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:09 2022] DELETE /identity/v3/users/db82d57a3af447f2a3045555a3c73029 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:10.349625 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:10.351703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:10.370978 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73e492-6bc1-47a8-b7a5-40d42da749d6 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:10.493140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:10.494260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:10.494730 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:10.495332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] PATH_INFO: `/v3/projects/411823cad8e14c259bfdd9c0816e2e47` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:10.496856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=411823cad8e14c259bfdd9c0816e2e47)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:10.499020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:10.609743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '411823cad8e14c259bfdd9c0816e2e47', 'request_id': 'req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:10.620994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '411823cad8e14c259bfdd9c0816e2e47', 'request_id': 'req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:10.631482 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '411823cad8e14c259bfdd9c0816e2e47', 'request_id': 'req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:10.648180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] The token cache is being invalidate because project 411823cad8e14c259bfdd9c0816e2e47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:10.648466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:10.649139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f80b1fd4-1bdb-46b2-be68-71b6f5f60db5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:10.650698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1275/2558] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:10 2022] DELETE /identity/v3/projects/411823cad8e14c259bfdd9c0816e2e47 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:10.659279 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32c12c-0ed7-49c3-b8d3-ed204740c2c7 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:10.659900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32c12c-0ed7-49c3-b8d3-ed204740c2c7 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:10.660640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32c12c-0ed7-49c3-b8d3-ed204740c2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:10.680568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-ae32c12c-0ed7-49c3-b8d3-ed204740c2c7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:10.681931 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-ae32c12c-0ed7-49c3-b8d3-ed204740c2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:10.765896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1284/2559] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:10.774179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b26121-f0f9-4255-b723-c42884b190d7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:10.774465 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b26121-f0f9-4255-b723-c42884b190d7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:10.774780 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b26121-f0f9-4255-b723-c42884b190d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:10.801605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a7b26121-f0f9-4255-b723-c42884b190d7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:10.805462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a7b26121-f0f9-4255-b723-c42884b190d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:10.810885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1276/2560] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:10.815881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32c12c-0ed7-49c3-b8d3-ed204740c2c7 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:10.828454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32c12c-0ed7-49c3-b8d3-ed204740c2c7 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:10.833247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199f98e-d6ed-4929-8e57-c898fef399ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:10.833918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199f98e-d6ed-4929-8e57-c898fef399ce admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:10.834194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199f98e-d6ed-4929-8e57-c898fef399ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:10.834585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199f98e-d6ed-4929-8e57-c898fef399ce admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:10.835259 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199f98e-d6ed-4929-8e57-c898fef399ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:10.835567 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199f98e-d6ed-4929-8e57-c898fef399ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:10.837298 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199f98e-d6ed-4929-8e57-c898fef399ce admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:10.859934 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1285/2561] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:04:10.865489 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b26121-f0f9-4255-b723-c42884b190d7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:10.875924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b26121-f0f9-4255-b723-c42884b190d7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:10.881278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417f692-9a3a-499b-93c2-6ff0f307b072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:10.882024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417f692-9a3a-499b-93c2-6ff0f307b072 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:10.882304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417f692-9a3a-499b-93c2-6ff0f307b072 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:10.882580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417f692-9a3a-499b-93c2-6ff0f307b072 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:10.883340 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417f692-9a3a-499b-93c2-6ff0f307b072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:10.885328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417f692-9a3a-499b-93c2-6ff0f307b072 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:10.927710 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1277/2562] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:10 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:04:10.929148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199f98e-d6ed-4929-8e57-c898fef399ce admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:10.943970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199f98e-d6ed-4929-8e57-c898fef399ce admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:11.024576 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c853c1-c78b-4028-a3dd-523aab6f2212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:11.025347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c853c1-c78b-4028-a3dd-523aab6f2212 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.025804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c853c1-c78b-4028-a3dd-523aab6f2212 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.026087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c853c1-c78b-4028-a3dd-523aab6f2212 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.027094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c853c1-c78b-4028-a3dd-523aab6f2212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:11.029151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c853c1-c78b-4028-a3dd-523aab6f2212 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:11.053099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1286/2563] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:10 2022] POST /identity/v3/users => generated 283 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:11.059033 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417f692-9a3a-499b-93c2-6ff0f307b072 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:11.074675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417f692-9a3a-499b-93c2-6ff0f307b072 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:11.081159 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14770a61-d08d-4d4a-8436-c73a2cdaecbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:11.081988 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14770a61-d08d-4d4a-8436-c73a2cdaecbd admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.082457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14770a61-d08d-4d4a-8436-c73a2cdaecbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.082862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14770a61-d08d-4d4a-8436-c73a2cdaecbd admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.083804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14770a61-d08d-4d4a-8436-c73a2cdaecbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:11.086082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14770a61-d08d-4d4a-8436-c73a2cdaecbd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:11.100006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1278/2564] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:11.104355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c853c1-c78b-4028-a3dd-523aab6f2212 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:11.116470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c853c1-c78b-4028-a3dd-523aab6f2212 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:11.121738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066eee9-680d-4e04-b88b-f279d39a3909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:11.122651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066eee9-680d-4e04-b88b-f279d39a3909 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.123602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066eee9-680d-4e04-b88b-f279d39a3909 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.123602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066eee9-680d-4e04-b88b-f279d39a3909 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.125459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066eee9-680d-4e04-b88b-f279d39a3909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:11.126418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066eee9-680d-4e04-b88b-f279d39a3909 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:11.145008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1287/2565] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:11.150749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14770a61-d08d-4d4a-8436-c73a2cdaecbd admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:11.166528 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14770a61-d08d-4d4a-8436-c73a2cdaecbd admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:11.172601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2c302-bf56-4299-9b45-a04c45aaf7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:11.173506 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2c302-bf56-4299-9b45-a04c45aaf7a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.173790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2c302-bf56-4299-9b45-a04c45aaf7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.174228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2c302-bf56-4299-9b45-a04c45aaf7a0 admin admin] PATH_INFO: `/v3/projects/be1a4af9209642c4aa95008b117d4c88/users/f6783d5b75da46108ccc97fdb27ba822/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.192525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2c302-bf56-4299-9b45-a04c45aaf7a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1a4af9209642c4aa95008b117d4c88, user_id=f6783d5b75da46108ccc97fdb27ba822, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:11.195304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2c302-bf56-4299-9b45-a04c45aaf7a0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:11.210523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1279/2566] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:11 2022] PUT /identity/v3/projects/be1a4af9209642c4aa95008b117d4c88/users/f6783d5b75da46108ccc97fdb27ba822/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:11.224078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15acb-cf2c-417e-9023-e7c98700819b None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.224643 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15acb-cf2c-417e-9023-e7c98700819b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.225120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15acb-cf2c-417e-9023-e7c98700819b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.276602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-5fb15acb-cf2c-417e-9023-e7c98700819b None None] MFA Rules not processed for user `f6783d5b75da46108ccc97fdb27ba822`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:11.279120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5fb15acb-cf2c-417e-9023-e7c98700819b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:11.366110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1288/2567] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:11.371344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2c302-bf56-4299-9b45-a04c45aaf7a0 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:11.453674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2c302-bf56-4299-9b45-a04c45aaf7a0 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:11.458652 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe48d-a3e2-41f1-82f3-dd42acfa8aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:11.459341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe48d-a3e2-41f1-82f3-dd42acfa8aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.459626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe48d-a3e2-41f1-82f3-dd42acfa8aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.459897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe48d-a3e2-41f1-82f3-dd42acfa8aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.460580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fe48d-a3e2-41f1-82f3-dd42acfa8aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:11.462501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fe48d-a3e2-41f1-82f3-dd42acfa8aa2 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:11.497654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1280/2568] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:11 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:11.502310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15acb-cf2c-417e-9023-e7c98700819b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:11.515917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15acb-cf2c-417e-9023-e7c98700819b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:11.623504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48a86c-6deb-49d5-918b-05f7f1e9b258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:11.625082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48a86c-6deb-49d5-918b-05f7f1e9b258 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.625562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48a86c-6deb-49d5-918b-05f7f1e9b258 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.625957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48a86c-6deb-49d5-918b-05f7f1e9b258 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.626610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48a86c-6deb-49d5-918b-05f7f1e9b258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:11.629451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48a86c-6deb-49d5-918b-05f7f1e9b258 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:11.658220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1289/2569] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:11 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:11.663274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe48d-a3e2-41f1-82f3-dd42acfa8aa2 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:11.678885 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe48d-a3e2-41f1-82f3-dd42acfa8aa2 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:11.685826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ceccac-f884-4d35-a924-54bf41898b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:11.687425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ceccac-f884-4d35-a924-54bf41898b23 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.687751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ceccac-f884-4d35-a924-54bf41898b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.687994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ceccac-f884-4d35-a924-54bf41898b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.688749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ceccac-f884-4d35-a924-54bf41898b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:11.690471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ceccac-f884-4d35-a924-54bf41898b23 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:11.706345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1281/2570] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:11.712405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48a86c-6deb-49d5-918b-05f7f1e9b258 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:11.727300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48a86c-6deb-49d5-918b-05f7f1e9b258 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:11.736649 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a64c10-8845-4b99-a452-35a77df09db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:11.738542 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a64c10-8845-4b99-a452-35a77df09db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.739166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a64c10-8845-4b99-a452-35a77df09db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.739687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a64c10-8845-4b99-a452-35a77df09db2 admin admin] PATH_INFO: `/v3/projects/985ba8133a6c4c23a0b336e3984cab31/users/29b49c49ec5d49a3b4bcdb0e351be66b/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.768697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a64c10-8845-4b99-a452-35a77df09db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985ba8133a6c4c23a0b336e3984cab31, user_id=29b49c49ec5d49a3b4bcdb0e351be66b, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:11.772780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a64c10-8845-4b99-a452-35a77df09db2 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:11.798379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1290/2571] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:11 2022] PUT /identity/v3/projects/985ba8133a6c4c23a0b336e3984cab31/users/29b49c49ec5d49a3b4bcdb0e351be66b/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:11.803369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ceccac-f884-4d35-a924-54bf41898b23 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:11.816822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ceccac-f884-4d35-a924-54bf41898b23 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:11.902021 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dee94-3dbe-406c-8ecf-fa0db45f8349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:11.902962 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dee94-3dbe-406c-8ecf-fa0db45f8349 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.903334 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dee94-3dbe-406c-8ecf-fa0db45f8349 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.903744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dee94-3dbe-406c-8ecf-fa0db45f8349 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.904624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5dee94-3dbe-406c-8ecf-fa0db45f8349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:11.909308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5dee94-3dbe-406c-8ecf-fa0db45f8349 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:11.926951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1282/2572] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:11.933081 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a64c10-8845-4b99-a452-35a77df09db2 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:11.945390 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a64c10-8845-4b99-a452-35a77df09db2 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:11.950616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e9392-b662-41fe-99db-76205be2efe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:11.951548 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e9392-b662-41fe-99db-76205be2efe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.951894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e9392-b662-41fe-99db-76205be2efe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.952104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e9392-b662-41fe-99db-76205be2efe0 admin admin] PATH_INFO: `/v3/projects/985ba8133a6c4c23a0b336e3984cab31/users/29b49c49ec5d49a3b4bcdb0e351be66b/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:11.954436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e9392-b662-41fe-99db-76205be2efe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985ba8133a6c4c23a0b336e3984cab31, user_id=29b49c49ec5d49a3b4bcdb0e351be66b, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:11.957944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e9392-b662-41fe-99db-76205be2efe0 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:11.971150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1291/2573] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:11 2022] PUT /identity/v3/projects/985ba8133a6c4c23a0b336e3984cab31/users/29b49c49ec5d49a3b4bcdb0e351be66b/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:11.984192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1ade7-668e-4c22-94b8-37fbe87ac19a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:11.984738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1ade7-668e-4c22-94b8-37fbe87ac19a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:11.985194 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1ade7-668e-4c22-94b8-37fbe87ac19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:12.040762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-bca1ade7-668e-4c22-94b8-37fbe87ac19a None None] MFA Rules not processed for user `29b49c49ec5d49a3b4bcdb0e351be66b`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:12.043743 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-bca1ade7-668e-4c22-94b8-37fbe87ac19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:12.164272 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1283/2574] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:12.177642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e9392-b662-41fe-99db-76205be2efe0 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:12.179443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0f4e9392-b662-41fe-99db-76205be2efe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:12.255397 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e9392-b662-41fe-99db-76205be2efe0 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:12.261023 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fe9db-913d-4b61-93f0-776361a88c65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:12.262151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fe9db-913d-4b61-93f0-776361a88c65 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:12.262915 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fe9db-913d-4b61-93f0-776361a88c65 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:12.263454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fe9db-913d-4b61-93f0-776361a88c65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:12.281044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fe9db-913d-4b61-93f0-776361a88c65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:12.284194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fe9db-913d-4b61-93f0-776361a88c65 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:12.295856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1292/2575] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:12.407347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1ade7-668e-4c22-94b8-37fbe87ac19a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:12.432326 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1ade7-668e-4c22-94b8-37fbe87ac19a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:12.439098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b02d83-346b-4d36-8a78-5575e9063852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:12.439848 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b02d83-346b-4d36-8a78-5575e9063852 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:12.440225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b02d83-346b-4d36-8a78-5575e9063852 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:12.440572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b02d83-346b-4d36-8a78-5575e9063852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:12.539776 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b02d83-346b-4d36-8a78-5575e9063852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:12.542024 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b02d83-346b-4d36-8a78-5575e9063852 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:12.556345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1284/2576] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:12.866830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fe9db-913d-4b61-93f0-776361a88c65 service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:12.868413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-186fe9db-913d-4b61-93f0-776361a88c65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:12.954606 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fe9db-913d-4b61-93f0-776361a88c65 service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:12.961532 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:12.962466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:12.962838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:12.963228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] PATH_INFO: `/v3/users/cd58b86b905e42aeb529904b0270636c` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:12.965082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd58b86b905e42aeb529904b0270636c)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:12.968210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:13.038350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd58b86b905e42aeb529904b0270636c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:13.051937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd58b86b905e42aeb529904b0270636c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:13.060094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd58b86b905e42aeb529904b0270636c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:13.061167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd58b86b905e42aeb529904b0270636c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:13.072026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd58b86b905e42aeb529904b0270636c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:13.088455 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1293/2577] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:12 2022] DELETE /identity/v3/users/cd58b86b905e42aeb529904b0270636c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:13.094414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9834b3-8721-4ac7-b553-e5d214de8a23 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:13.095128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9834b3-8721-4ac7-b553-e5d214de8a23 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:13.095681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9834b3-8721-4ac7-b553-e5d214de8a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:13.122810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-6c9834b3-8721-4ac7-b553-e5d214de8a23 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:13.125102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6c9834b3-8721-4ac7-b553-e5d214de8a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:13.214448 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:13.216753 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:13.251919 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1285/2578] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:13.262398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9834b3-8721-4ac7-b553-e5d214de8a23 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:13.264112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6c9834b3-8721-4ac7-b553-e5d214de8a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:13.321927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005db82d-ad15-4ba6-bb5f-2868803c18dc admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:13.331494 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5483a-d648-4960-aa2d-491e10c0e26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:13.332176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5483a-d648-4960-aa2d-491e10c0e26d service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:13.332462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5483a-d648-4960-aa2d-491e10c0e26d service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:13.332742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5483a-d648-4960-aa2d-491e10c0e26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:13.335438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7ed5483a-d648-4960-aa2d-491e10c0e26d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:13.352126 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5483a-d648-4960-aa2d-491e10c0e26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:13.354866 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5483a-d648-4960-aa2d-491e10c0e26d service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:13.357132 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9834b3-8721-4ac7-b553-e5d214de8a23 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:13.364105 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c327c4-546c-4f53-af4c-a131cbc8c3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:13.365066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c327c4-546c-4f53-af4c-a131cbc8c3db service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:13.365390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c327c4-546c-4f53-af4c-a131cbc8c3db service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:13.365758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c327c4-546c-4f53-af4c-a131cbc8c3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:13.374291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1294/2579] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:04:13.381435 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c327c4-546c-4f53-af4c-a131cbc8c3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:13.383137 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c327c4-546c-4f53-af4c-a131cbc8c3db service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:13.395158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1286/2580] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:04:14.025568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5483a-d648-4960-aa2d-491e10c0e26d service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:14.027926 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7ed5483a-d648-4960-aa2d-491e10c0e26d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:14.063087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5483a-d648-4960-aa2d-491e10c0e26d service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:14.070953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:14.072442 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:14.073918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:14.074472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] PATH_INFO: `/v3/projects/484e74418bcc4b398bad5b2de6ff220c` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:14.077979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=484e74418bcc4b398bad5b2de6ff220c)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:14.081836 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:14.185350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484e74418bcc4b398bad5b2de6ff220c', 'request_id': 'req-b05e5080-bb3a-4fbd-86fa-095687e6abc6'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:14.196800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484e74418bcc4b398bad5b2de6ff220c', 'request_id': 'req-b05e5080-bb3a-4fbd-86fa-095687e6abc6'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:14.214807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484e74418bcc4b398bad5b2de6ff220c', 'request_id': 'req-b05e5080-bb3a-4fbd-86fa-095687e6abc6'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:14.236025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] The token cache is being invalidate because project 484e74418bcc4b398bad5b2de6ff220c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:14.236399 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:14.237247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b05e5080-bb3a-4fbd-86fa-095687e6abc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:14.240022 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1295/2581] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:14 2022] DELETE /identity/v3/projects/484e74418bcc4b398bad5b2de6ff220c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:14.247613 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104898-5a9c-4e3d-b673-e5a35f596c6b None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:14.248394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104898-5a9c-4e3d-b673-e5a35f596c6b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:14.248394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104898-5a9c-4e3d-b673-e5a35f596c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:14.290225 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-aa104898-5a9c-4e3d-b673-e5a35f596c6b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:14.293579 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-aa104898-5a9c-4e3d-b673-e5a35f596c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:14.428231 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1287/2582] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:14.441035 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aca06-5fd3-407f-af19-f1ff4efbae9b None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:14.441812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aca06-5fd3-407f-af19-f1ff4efbae9b None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:14.441812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963aca06-5fd3-407f-af19-f1ff4efbae9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:14.464573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-963aca06-5fd3-407f-af19-f1ff4efbae9b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:14.466050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-963aca06-5fd3-407f-af19-f1ff4efbae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:14.479333 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1296/2583] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:14.500210 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104898-5a9c-4e3d-b673-e5a35f596c6b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:14.522250 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104898-5a9c-4e3d-b673-e5a35f596c6b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:14.537123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ce4b0-f76e-4cc9-90c8-7e5e40fb871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:14.538338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce4b0-f76e-4cc9-90c8-7e5e40fb871b admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:14.538987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce4b0-f76e-4cc9-90c8-7e5e40fb871b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:14.539400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce4b0-f76e-4cc9-90c8-7e5e40fb871b admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:14.541007 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ce4b0-f76e-4cc9-90c8-7e5e40fb871b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:14.541907 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ce4b0-f76e-4cc9-90c8-7e5e40fb871b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:14.544897 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ce4b0-f76e-4cc9-90c8-7e5e40fb871b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:14.570157 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1288/2584] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 14:04:14.575783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963aca06-5fd3-407f-af19-f1ff4efbae9b None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:14.594157 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963aca06-5fd3-407f-af19-f1ff4efbae9b None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:14.604314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b066f-dabc-4d8b-8573-2e1892deda2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:14.605173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b066f-dabc-4d8b-8573-2e1892deda2e admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:14.605698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b066f-dabc-4d8b-8573-2e1892deda2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:14.606058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b066f-dabc-4d8b-8573-2e1892deda2e admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:14.607072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b066f-dabc-4d8b-8573-2e1892deda2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:14.610144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b066f-dabc-4d8b-8573-2e1892deda2e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:14.662257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1297/2585] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:14 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:14.668079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ce4b0-f76e-4cc9-90c8-7e5e40fb871b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:14.682178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ce4b0-f76e-4cc9-90c8-7e5e40fb871b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:14.795410 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdb8d2-5b29-45ef-ad3c-4227e37a32e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:14.796619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdb8d2-5b29-45ef-ad3c-4227e37a32e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:14.797299 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdb8d2-5b29-45ef-ad3c-4227e37a32e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:14.797797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdb8d2-5b29-45ef-ad3c-4227e37a32e8 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:14.799166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdb8d2-5b29-45ef-ad3c-4227e37a32e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:14.803308 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdb8d2-5b29-45ef-ad3c-4227e37a32e8 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:14.829052 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1289/2586] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:14 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:14.842218 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b066f-dabc-4d8b-8573-2e1892deda2e admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:14.855834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b066f-dabc-4d8b-8573-2e1892deda2e admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:14.861254 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d06f69-ea5e-45de-8a95-a792a27e9d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:14.862040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d06f69-ea5e-45de-8a95-a792a27e9d0a admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:14.862328 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d06f69-ea5e-45de-8a95-a792a27e9d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:14.862602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d06f69-ea5e-45de-8a95-a792a27e9d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:14.863416 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d06f69-ea5e-45de-8a95-a792a27e9d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:14.865246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d06f69-ea5e-45de-8a95-a792a27e9d0a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:14.880766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1298/2587] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:14.887468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdb8d2-5b29-45ef-ad3c-4227e37a32e8 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:14.909135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdb8d2-5b29-45ef-ad3c-4227e37a32e8 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:14.922790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a928f-627d-435e-8804-aba2d795fb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:14.923727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a928f-627d-435e-8804-aba2d795fb87 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:14.924155 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a928f-627d-435e-8804-aba2d795fb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:14.924580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a928f-627d-435e-8804-aba2d795fb87 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:14.925735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a928f-627d-435e-8804-aba2d795fb87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:14.927689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a928f-627d-435e-8804-aba2d795fb87 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:14.941968 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1290/2588] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:04:14.948562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d06f69-ea5e-45de-8a95-a792a27e9d0a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:14.964468 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d06f69-ea5e-45de-8a95-a792a27e9d0a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:14.969914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bf7d3-6462-4871-be50-cb3cb693f2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:14.970984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf7d3-6462-4871-be50-cb3cb693f2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:14.971405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf7d3-6462-4871-be50-cb3cb693f2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:14.971659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf7d3-6462-4871-be50-cb3cb693f2ec admin admin] PATH_INFO: `/v3/projects/673435f01c47460bae84e4f31643f66d/users/3bbfb51ebb5f4553a9ef7b70becf1a9a/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:15.001014 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bf7d3-6462-4871-be50-cb3cb693f2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=673435f01c47460bae84e4f31643f66d, user_id=3bbfb51ebb5f4553a9ef7b70becf1a9a, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:15.004453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bf7d3-6462-4871-be50-cb3cb693f2ec admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:15.029539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1299/2589] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:14 2022] PUT /identity/v3/projects/673435f01c47460bae84e4f31643f66d/users/3bbfb51ebb5f4553a9ef7b70becf1a9a/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:15.037013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43b3c6-ca3d-4d82-9cc7-e48486c9f202 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:15.037013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43b3c6-ca3d-4d82-9cc7-e48486c9f202 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:15.037013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43b3c6-ca3d-4d82-9cc7-e48486c9f202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:15.063221 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-4e43b3c6-ca3d-4d82-9cc7-e48486c9f202 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:15.064846 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4e43b3c6-ca3d-4d82-9cc7-e48486c9f202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:15.175405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1291/2590] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:15.189531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bf7d3-6462-4871-be50-cb3cb693f2ec admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:15.193535 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-966bf7d3-6462-4871-be50-cb3cb693f2ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:15.293447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bf7d3-6462-4871-be50-cb3cb693f2ec admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:15.299189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc8cff-e9c6-425e-82ba-b0a1b2f002be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:15.299911 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc8cff-e9c6-425e-82ba-b0a1b2f002be service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:15.300197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc8cff-e9c6-425e-82ba-b0a1b2f002be service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:15.300469 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc8cff-e9c6-425e-82ba-b0a1b2f002be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:15.316223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc8cff-e9c6-425e-82ba-b0a1b2f002be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:15.318149 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc8cff-e9c6-425e-82ba-b0a1b2f002be service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:15.331315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1300/2591] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:15.856467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43b3c6-ca3d-4d82-9cc7-e48486c9f202 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:15.862056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4e43b3c6-ca3d-4d82-9cc7-e48486c9f202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:15.893620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43b3c6-ca3d-4d82-9cc7-e48486c9f202 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:15.917234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:15.918212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:15.918526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:15.918832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] PATH_INFO: `/v3/users/f6783d5b75da46108ccc97fdb27ba822` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:15.921912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6783d5b75da46108ccc97fdb27ba822)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:15.924112 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:16.033303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6783d5b75da46108ccc97fdb27ba822'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:16.034034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6783d5b75da46108ccc97fdb27ba822'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:16.077594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6783d5b75da46108ccc97fdb27ba822'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:16.078740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6783d5b75da46108ccc97fdb27ba822'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:16.093796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6783d5b75da46108ccc97fdb27ba822'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:16.116505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1292/2592] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:15 2022] DELETE /identity/v3/users/f6783d5b75da46108ccc97fdb27ba822 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:16.123362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c56a13-6177-43f0-b5a1-3b57d2b1ec68 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:16.125922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c56a13-6177-43f0-b5a1-3b57d2b1ec68 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:16.127139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c56a13-6177-43f0-b5a1-3b57d2b1ec68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:16.154524 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-32c56a13-6177-43f0-b5a1-3b57d2b1ec68 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:16.156772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-32c56a13-6177-43f0-b5a1-3b57d2b1ec68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:16.315942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1301/2593] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:16.335442 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:16.338812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:16.445369 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220b4d3-7b7b-433a-88f1-494b3499c1dc admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:16.455136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ed3e0-8841-4d37-b34c-42e609a638ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:16.457734 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ed3e0-8841-4d37-b34c-42e609a638ff service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:16.458301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ed3e0-8841-4d37-b34c-42e609a638ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:16.458900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ed3e0-8841-4d37-b34c-42e609a638ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:16.481289 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ed3e0-8841-4d37-b34c-42e609a638ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:16.486219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ed3e0-8841-4d37-b34c-42e609a638ff service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:16.508014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1293/2594] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:16.993150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c56a13-6177-43f0-b5a1-3b57d2b1ec68 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:16.995732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-32c56a13-6177-43f0-b5a1-3b57d2b1ec68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:17.037815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c56a13-6177-43f0-b5a1-3b57d2b1ec68 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:17.048713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:17.049851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:17.050503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:17.051344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] PATH_INFO: `/v3/projects/be1a4af9209642c4aa95008b117d4c88` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:17.053515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1a4af9209642c4aa95008b117d4c88)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:17.057572 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:17.138808 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1a4af9209642c4aa95008b117d4c88', 'request_id': 'req-6da38e0e-b6af-44d7-ab8d-369171281663'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:17.146350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1a4af9209642c4aa95008b117d4c88', 'request_id': 'req-6da38e0e-b6af-44d7-ab8d-369171281663'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:17.163194 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1a4af9209642c4aa95008b117d4c88', 'request_id': 'req-6da38e0e-b6af-44d7-ab8d-369171281663'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:17.193332 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] The token cache is being invalidate because project be1a4af9209642c4aa95008b117d4c88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:17.193616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:17.194572 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:17.196583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1302/2595] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:16 2022] DELETE /identity/v3/projects/be1a4af9209642c4aa95008b117d4c88 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:17.201801 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ed3e0-8841-4d37-b34c-42e609a638ff service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:17.217284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ed3e0-8841-4d37-b34c-42e609a638ff service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:17.324695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:17.325891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:17.326242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:17.326630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] PATH_INFO: `/v3/users/29b49c49ec5d49a3b4bcdb0e351be66b` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:17.329082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b49c49ec5d49a3b4bcdb0e351be66b)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:17.331681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:17.411072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b49c49ec5d49a3b4bcdb0e351be66b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:17.412468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b49c49ec5d49a3b4bcdb0e351be66b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:17.440783 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b49c49ec5d49a3b4bcdb0e351be66b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:17.442266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b49c49ec5d49a3b4bcdb0e351be66b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:17.462209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-c5139f5e-fda2-435c-a76a-5c59cc7dcd9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b49c49ec5d49a3b4bcdb0e351be66b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:17.487973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1294/2596] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:17 2022] DELETE /identity/v3/users/29b49c49ec5d49a3b4bcdb0e351be66b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:18.639082 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:18.645827 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:18.673491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da38e0e-b6af-44d7-ab8d-369171281663 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:18.850150 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:18.851270 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:18.851723 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:18.852322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] PATH_INFO: `/v3/projects/985ba8133a6c4c23a0b336e3984cab31` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:18.857286 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] RBAC: Authorizing `identity:delete_project(project_id=985ba8133a6c4c23a0b336e3984cab31)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:18.860344 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:18.937313 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985ba8133a6c4c23a0b336e3984cab31', 'request_id': 'req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:18.947350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985ba8133a6c4c23a0b336e3984cab31', 'request_id': 'req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:18.960825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985ba8133a6c4c23a0b336e3984cab31', 'request_id': 'req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:18.998979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] The token cache is being invalidate because project 985ba8133a6c4c23a0b336e3984cab31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:18.999891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:19.002137 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-6398a4fd-04c3-4deb-aa5e-43ec48b1557d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:19.005572 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1303/2597] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:18 2022] DELETE /identity/v3/projects/985ba8133a6c4c23a0b336e3984cab31 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:19.013654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83692b-480b-48fb-b9c3-63e1696e6367 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:19.014422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83692b-480b-48fb-b9c3-63e1696e6367 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:19.014422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83692b-480b-48fb-b9c3-63e1696e6367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:19.045319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-9c83692b-480b-48fb-b9c3-63e1696e6367 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:19.054275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9c83692b-480b-48fb-b9c3-63e1696e6367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:19.179936 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1295/2598] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:19.187871 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28aee-dc6c-488c-8d4e-cf590c358c5e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:19.188207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28aee-dc6c-488c-8d4e-cf590c358c5e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:19.188427 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28aee-dc6c-488c-8d4e-cf590c358c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:19.225656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-40e28aee-dc6c-488c-8d4e-cf590c358c5e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:19.227224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-40e28aee-dc6c-488c-8d4e-cf590c358c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:19.244167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1304/2599] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:19.250696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83692b-480b-48fb-b9c3-63e1696e6367 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:19.278197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83692b-480b-48fb-b9c3-63e1696e6367 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:19.294499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a990bf-11ad-4df5-a17e-b2226e82d772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:19.300315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a990bf-11ad-4df5-a17e-b2226e82d772 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:19.302152 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a990bf-11ad-4df5-a17e-b2226e82d772 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:19.303028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a990bf-11ad-4df5-a17e-b2226e82d772 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:19.304527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a990bf-11ad-4df5-a17e-b2226e82d772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:19.305174 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a990bf-11ad-4df5-a17e-b2226e82d772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:19.308147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a990bf-11ad-4df5-a17e-b2226e82d772 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:19.341053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1296/2600] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 14:04:19.347697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e28aee-dc6c-488c-8d4e-cf590c358c5e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:19.371278 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e28aee-dc6c-488c-8d4e-cf590c358c5e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:19.441132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf4772-60b0-41f8-8b31-9a672fb15c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:19.442180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf4772-60b0-41f8-8b31-9a672fb15c3f admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:19.442582 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf4772-60b0-41f8-8b31-9a672fb15c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:19.443056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf4772-60b0-41f8-8b31-9a672fb15c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:19.446367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf4772-60b0-41f8-8b31-9a672fb15c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:19.453443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf4772-60b0-41f8-8b31-9a672fb15c3f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:19.507413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1305/2601] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:19 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:19.512996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a990bf-11ad-4df5-a17e-b2226e82d772 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:19.541354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a990bf-11ad-4df5-a17e-b2226e82d772 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:19.715836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effe95d-2d63-459e-b46c-d516b1263c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:19.717120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effe95d-2d63-459e-b46c-d516b1263c8a admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:19.723346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effe95d-2d63-459e-b46c-d516b1263c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:19.723346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effe95d-2d63-459e-b46c-d516b1263c8a admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:19.723346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effe95d-2d63-459e-b46c-d516b1263c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:19.732275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effe95d-2d63-459e-b46c-d516b1263c8a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:19.800847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1297/2602] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:19 2022] POST /identity/v3/users => generated 294 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:19.802609 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf4772-60b0-41f8-8b31-9a672fb15c3f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:19.859733 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf4772-60b0-41f8-8b31-9a672fb15c3f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:19.869918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8812b-9576-4cd9-8935-a64e9b24ee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:19.871040 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8812b-9576-4cd9-8935-a64e9b24ee34 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:19.872071 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8812b-9576-4cd9-8935-a64e9b24ee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:19.872528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8812b-9576-4cd9-8935-a64e9b24ee34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:19.873503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8812b-9576-4cd9-8935-a64e9b24ee34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:19.875777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effe95d-2d63-459e-b46c-d516b1263c8a admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:19.876459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8812b-9576-4cd9-8935-a64e9b24ee34 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:19.879603 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8effe95d-2d63-459e-b46c-d516b1263c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:19.895677 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1306/2603] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:19.900355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8812b-9576-4cd9-8935-a64e9b24ee34 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:19.924029 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8812b-9576-4cd9-8935-a64e9b24ee34 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:19.926511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effe95d-2d63-459e-b46c-d516b1263c8a admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:19.932036 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d369063-b1e4-47d0-89a6-8328128969c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:19.933087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d369063-b1e4-47d0-89a6-8328128969c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:19.933724 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d369063-b1e4-47d0-89a6-8328128969c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:19.934417 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d369063-b1e4-47d0-89a6-8328128969c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:19.935694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:19.936398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d369063-b1e4-47d0-89a6-8328128969c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:19.939382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:19.940045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:19.940674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] PATH_INFO: `/v3/users/8b2980196e0f4fafa5131dd5de02ac75` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:19.941925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d369063-b1e4-47d0-89a6-8328128969c4 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:19.943480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2980196e0f4fafa5131dd5de02ac75)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:19.947723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:19.964015 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1307/2604] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:19.968852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d369063-b1e4-47d0-89a6-8328128969c4 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:19.986248 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d369063-b1e4-47d0-89a6-8328128969c4 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:20.001177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66fc53-9324-43d0-9ea1-ac90fba36cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:20.005799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66fc53-9324-43d0-9ea1-ac90fba36cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:20.006402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66fc53-9324-43d0-9ea1-ac90fba36cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:20.007196 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66fc53-9324-43d0-9ea1-ac90fba36cb0 admin admin] PATH_INFO: `/v3/projects/d11e9192133c492ab084516adc55f181/users/0c98e3f5362a4d60abd3ddf4e1e65071/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:20.045820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66fc53-9324-43d0-9ea1-ac90fba36cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11e9192133c492ab084516adc55f181, user_id=0c98e3f5362a4d60abd3ddf4e1e65071, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:20.052010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66fc53-9324-43d0-9ea1-ac90fba36cb0 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:20.075828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2980196e0f4fafa5131dd5de02ac75'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:20.079109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2980196e0f4fafa5131dd5de02ac75'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:20.084439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1308/2605] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:19 2022] PUT /identity/v3/projects/d11e9192133c492ab084516adc55f181/users/0c98e3f5362a4d60abd3ddf4e1e65071/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:20.106028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56554bd-fb33-40de-95ad-51a4f79ffc81 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:20.106525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56554bd-fb33-40de-95ad-51a4f79ffc81 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:20.107070 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56554bd-fb33-40de-95ad-51a4f79ffc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:20.131319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2980196e0f4fafa5131dd5de02ac75'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:20.132733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2980196e0f4fafa5131dd5de02ac75'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:20.151190 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2980196e0f4fafa5131dd5de02ac75'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:20.176963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-c56554bd-fb33-40de-95ad-51a4f79ffc81 None None] MFA Rules not processed for user `0c98e3f5362a4d60abd3ddf4e1e65071`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:20.182634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c56554bd-fb33-40de-95ad-51a4f79ffc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:20.192999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1298/2606] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:19 2022] DELETE /identity/v3/users/8b2980196e0f4fafa5131dd5de02ac75 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:20.389394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1309/2607] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:20.396524 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:20.399779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:20.590218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9244590-b4d1-4e9f-8ac4-a196c35e1c40 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:20.609685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56554bd-fb33-40de-95ad-51a4f79ffc81 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:20.610275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c1ca5-8f47-4c5c-be00-087d7d71633d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:20.611547 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1ca5-8f47-4c5c-be00-087d7d71633d admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:20.612095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1ca5-8f47-4c5c-be00-087d7d71633d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:20.612650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1ca5-8f47-4c5c-be00-087d7d71633d admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:20.612840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c56554bd-fb33-40de-95ad-51a4f79ffc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:20.620064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c1ca5-8f47-4c5c-be00-087d7d71633d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:20.622364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c1ca5-8f47-4c5c-be00-087d7d71633d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:20.663978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56554bd-fb33-40de-95ad-51a4f79ffc81 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:20.691927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:20.694690 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:20.695002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:20.695284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] PATH_INFO: `/v3/projects/203eb74b5a2c444aac67940d775ecfc3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:20.698167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1299/2608] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:20 2022] POST /identity/v3/projects => generated 385 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:20.699219 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=203eb74b5a2c444aac67940d775ecfc3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:20.704228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c1ca5-8f47-4c5c-be00-087d7d71633d admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:20.705825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:20.885669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203eb74b5a2c444aac67940d775ecfc3', 'request_id': 'req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:20.894271 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203eb74b5a2c444aac67940d775ecfc3', 'request_id': 'req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:20.910264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203eb74b5a2c444aac67940d775ecfc3', 'request_id': 'req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:20.918233 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c1ca5-8f47-4c5c-be00-087d7d71633d admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:20.942021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] The token cache is being invalidate because project 203eb74b5a2c444aac67940d775ecfc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:20.942238 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:20.947159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-ab058c45-cbf4-4da8-bd24-024dbe3c17f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:20.958998 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1310/2609] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:20 2022] DELETE /identity/v3/projects/203eb74b5a2c444aac67940d775ecfc3 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:20.962130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d49fb-0ae8-4f0a-806c-cbd8a363d8a3 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:20.962130 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d49fb-0ae8-4f0a-806c-cbd8a363d8a3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:20.962514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d49fb-0ae8-4f0a-806c-cbd8a363d8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.023707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-515d49fb-0ae8-4f0a-806c-cbd8a363d8a3 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:21.025032 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-515d49fb-0ae8-4f0a-806c-cbd8a363d8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:21.096753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f5d2c-aaf0-458e-8e9f-4ca2ed2ac0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:21.099140 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f5d2c-aaf0-458e-8e9f-4ca2ed2ac0fb admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:21.099930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f5d2c-aaf0-458e-8e9f-4ca2ed2ac0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:21.100633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f5d2c-aaf0-458e-8e9f-4ca2ed2ac0fb admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.102560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f5d2c-aaf0-458e-8e9f-4ca2ed2ac0fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:21.106352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f5d2c-aaf0-458e-8e9f-4ca2ed2ac0fb admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:21.195001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1300/2610] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:20 2022] POST /identity/v3/users => generated 299 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:21.201042 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f5d2c-aaf0-458e-8e9f-4ca2ed2ac0fb admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:21.204651 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1e6f5d2c-aaf0-458e-8e9f-4ca2ed2ac0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:21.226288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1311/2611] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:21.243569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab93fc-59a4-4713-9e2a-eda58f16f3e1 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:21.244076 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab93fc-59a4-4713-9e2a-eda58f16f3e1 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:21.244614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab93fc-59a4-4713-9e2a-eda58f16f3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.249198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f5d2c-aaf0-458e-8e9f-4ca2ed2ac0fb admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:21.259128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1772f9-eb7a-469e-8520-8f283a2942ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:21.262123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1772f9-eb7a-469e-8520-8f283a2942ac admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:21.263598 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1772f9-eb7a-469e-8520-8f283a2942ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:21.264354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1772f9-eb7a-469e-8520-8f283a2942ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.266774 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1772f9-eb7a-469e-8520-8f283a2942ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:21.270421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1772f9-eb7a-469e-8520-8f283a2942ac admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:21.304868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1301/2612] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:04:21.308375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-acab93fc-59a4-4713-9e2a-eda58f16f3e1 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:21.308709 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1772f9-eb7a-469e-8520-8f283a2942ac admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:21.311185 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-acab93fc-59a4-4713-9e2a-eda58f16f3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:21.328092 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1772f9-eb7a-469e-8520-8f283a2942ac admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:21.337570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec933c4-db32-4659-923e-5f2df1a9f497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:21.339769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec933c4-db32-4659-923e-5f2df1a9f497 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:21.340003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1312/2613] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:21.340438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec933c4-db32-4659-923e-5f2df1a9f497 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:21.341036 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec933c4-db32-4659-923e-5f2df1a9f497 admin admin] PATH_INFO: `/v3/projects/95f55d0ed5c4457597132319d6a22d47/users/b1f29500ea154582a5e7c028f4beac1a/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.342957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab93fc-59a4-4713-9e2a-eda58f16f3e1 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:21.363575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec933c4-db32-4659-923e-5f2df1a9f497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f55d0ed5c4457597132319d6a22d47, user_id=b1f29500ea154582a5e7c028f4beac1a, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:21.367335 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec933c4-db32-4659-923e-5f2df1a9f497 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:21.368501 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab93fc-59a4-4713-9e2a-eda58f16f3e1 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:21.387914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57603700-0f98-4f0b-8fb2-65b5180b7935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:21.392893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57603700-0f98-4f0b-8fb2-65b5180b7935 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:21.393404 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57603700-0f98-4f0b-8fb2-65b5180b7935 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:21.393893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57603700-0f98-4f0b-8fb2-65b5180b7935 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.399095 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57603700-0f98-4f0b-8fb2-65b5180b7935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:21.400817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57603700-0f98-4f0b-8fb2-65b5180b7935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:21.402950 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57603700-0f98-4f0b-8fb2-65b5180b7935 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:21.410374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1302/2614] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:21 2022] PUT /identity/v3/projects/95f55d0ed5c4457597132319d6a22d47/users/b1f29500ea154582a5e7c028f4beac1a/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:21.412237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec933c4-db32-4659-923e-5f2df1a9f497 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:21.451710 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1313/2615] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:04:21.456521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57603700-0f98-4f0b-8fb2-65b5180b7935 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:21.477758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57603700-0f98-4f0b-8fb2-65b5180b7935 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:21.518260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec933c4-db32-4659-923e-5f2df1a9f497 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:21.531863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ec278-68a8-4651-aa36-4c530db2dd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:21.532975 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ec278-68a8-4651-aa36-4c530db2dd89 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:21.539510 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ec278-68a8-4651-aa36-4c530db2dd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:21.540011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ec278-68a8-4651-aa36-4c530db2dd89 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.541285 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ec278-68a8-4651-aa36-4c530db2dd89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:21.543820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ec278-68a8-4651-aa36-4c530db2dd89 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:21.568923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1303/2616] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:21.581613 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ec278-68a8-4651-aa36-4c530db2dd89 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:21.605855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ec278-68a8-4651-aa36-4c530db2dd89 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:21.618363 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4beb36-1cab-43a2-ab55-16a540143b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:21.619486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4beb36-1cab-43a2-ab55-16a540143b0e admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:21.619860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4beb36-1cab-43a2-ab55-16a540143b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:21.624164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4beb36-1cab-43a2-ab55-16a540143b0e admin admin] PATH_INFO: `/v3/projects/95f55d0ed5c4457597132319d6a22d47/users/b1f29500ea154582a5e7c028f4beac1a/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.627050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc7109-027a-4ece-925f-014a454f2262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:21.627332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4beb36-1cab-43a2-ab55-16a540143b0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f55d0ed5c4457597132319d6a22d47, user_id=b1f29500ea154582a5e7c028f4beac1a, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:21.628131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc7109-027a-4ece-925f-014a454f2262 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:21.628599 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc7109-027a-4ece-925f-014a454f2262 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:21.629051 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc7109-027a-4ece-925f-014a454f2262 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.630400 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4beb36-1cab-43a2-ab55-16a540143b0e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:21.631301 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc7109-027a-4ece-925f-014a454f2262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:21.634447 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc7109-027a-4ece-925f-014a454f2262 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:21.666955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1304/2617] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:21 2022] PUT /identity/v3/projects/95f55d0ed5c4457597132319d6a22d47/users/b1f29500ea154582a5e7c028f4beac1a/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:21.668808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a6bf7-b4cc-4ce1-a714-2c425456d198 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:21.670631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a6bf7-b4cc-4ce1-a714-2c425456d198 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:21.670631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a6bf7-b4cc-4ce1-a714-2c425456d198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.705159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1314/2618] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:21 2022] POST /identity/v3/projects => generated 373 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:21.710797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc7109-027a-4ece-925f-014a454f2262 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:21.720455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-352a6bf7-b4cc-4ce1-a714-2c425456d198 None None] MFA Rules not processed for user `b1f29500ea154582a5e7c028f4beac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:21.720752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-352a6bf7-b4cc-4ce1-a714-2c425456d198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:21.841902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1305/2619] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:21.865116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a6bf7-b4cc-4ce1-a714-2c425456d198 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:21.867290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-352a6bf7-b4cc-4ce1-a714-2c425456d198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:21.881165 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc7109-027a-4ece-925f-014a454f2262 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:21.889830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf48a82-e608-4c77-b19a-3e76849e1743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:21.895952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf48a82-e608-4c77-b19a-3e76849e1743 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:21.896823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf48a82-e608-4c77-b19a-3e76849e1743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:21.897760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf48a82-e608-4c77-b19a-3e76849e1743 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:21.901269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf48a82-e608-4c77-b19a-3e76849e1743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:21.906142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf48a82-e608-4c77-b19a-3e76849e1743 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:21.966343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1315/2620] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:21 2022] POST /identity/v3/users => generated 287 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:21.968977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf48a82-e608-4c77-b19a-3e76849e1743 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:21.995026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf48a82-e608-4c77-b19a-3e76849e1743 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:22.013695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb322657-4afb-4d2c-9351-24a375c92fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:22.014388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb322657-4afb-4d2c-9351-24a375c92fcf admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:22.014665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb322657-4afb-4d2c-9351-24a375c92fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:22.015514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb322657-4afb-4d2c-9351-24a375c92fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:22.016272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb322657-4afb-4d2c-9351-24a375c92fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:22.019522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb322657-4afb-4d2c-9351-24a375c92fcf admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:22.041927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1316/2621] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:22.046772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a6bf7-b4cc-4ce1-a714-2c425456d198 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:22.047894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb322657-4afb-4d2c-9351-24a375c92fcf admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:22.054817 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91535d3b-b8e9-4071-8c7d-ba4744bf304b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:22.057637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91535d3b-b8e9-4071-8c7d-ba4744bf304b service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:22.060084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91535d3b-b8e9-4071-8c7d-ba4744bf304b service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:22.060493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91535d3b-b8e9-4071-8c7d-ba4744bf304b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:22.078758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb322657-4afb-4d2c-9351-24a375c92fcf admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:22.093193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91535d3b-b8e9-4071-8c7d-ba4744bf304b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:22.101828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91535d3b-b8e9-4071-8c7d-ba4744bf304b service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:22.103047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6599ad-408f-4f65-9493-b0db92e9de03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:22.104527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6599ad-408f-4f65-9493-b0db92e9de03 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:22.104937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6599ad-408f-4f65-9493-b0db92e9de03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:22.107815 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6599ad-408f-4f65-9493-b0db92e9de03 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:22.111300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6599ad-408f-4f65-9493-b0db92e9de03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:22.124339 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1306/2622] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:22.125878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6599ad-408f-4f65-9493-b0db92e9de03 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:22.180190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1317/2623] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:04:22.183190 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91535d3b-b8e9-4071-8c7d-ba4744bf304b service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:22.231601 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91535d3b-b8e9-4071-8c7d-ba4744bf304b service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:22.234693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cc6c5-5e08-4fc4-937a-561130bf2652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:22.236293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cc6c5-5e08-4fc4-937a-561130bf2652 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:22.236513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cc6c5-5e08-4fc4-937a-561130bf2652 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:22.236989 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cc6c5-5e08-4fc4-937a-561130bf2652 admin admin] PATH_INFO: `/v3/projects/f6e66a5c52a440f6b64881277a23328c/users/f9b67061bf654460a19409dc1ec93b76/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:22.292470 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4cc6c5-5e08-4fc4-937a-561130bf2652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e66a5c52a440f6b64881277a23328c, user_id=f9b67061bf654460a19409dc1ec93b76, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:22.297413 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4cc6c5-5e08-4fc4-937a-561130bf2652 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:22.302879 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6599ad-408f-4f65-9493-b0db92e9de03 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:22.331476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6599ad-408f-4f65-9493-b0db92e9de03 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:22.358061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1307/2624] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:22 2022] PUT /identity/v3/projects/f6e66a5c52a440f6b64881277a23328c/users/f9b67061bf654460a19409dc1ec93b76/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:22.384607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367b1fe-a79f-4682-8515-b883f81e2b60 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:22.385368 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367b1fe-a79f-4682-8515-b883f81e2b60 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:22.385908 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367b1fe-a79f-4682-8515-b883f81e2b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:22.450239 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-3367b1fe-a79f-4682-8515-b883f81e2b60 None None] MFA Rules not processed for user `f9b67061bf654460a19409dc1ec93b76`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:22.451622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-3367b1fe-a79f-4682-8515-b883f81e2b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:22.488781 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:22.490413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:22.490853 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:22.491472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] PATH_INFO: `/v3/users/0c98e3f5362a4d60abd3ddf4e1e65071` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:22.493933 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c98e3f5362a4d60abd3ddf4e1e65071)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:22.497363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:22.601724 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c98e3f5362a4d60abd3ddf4e1e65071'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:22.621001 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c98e3f5362a4d60abd3ddf4e1e65071'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:22.635894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c98e3f5362a4d60abd3ddf4e1e65071'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:22.636245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c98e3f5362a4d60abd3ddf4e1e65071'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:22.645735 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1308/2625] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:22.657933 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-0acde18d-7814-4d0d-9634-5fed9b684ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c98e3f5362a4d60abd3ddf4e1e65071'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:22.660061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367b1fe-a79f-4682-8515-b883f81e2b60 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:22.661883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-3367b1fe-a79f-4682-8515-b883f81e2b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:22.676739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1318/2626] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:22 2022] DELETE /identity/v3/users/0c98e3f5362a4d60abd3ddf4e1e65071 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:22.687347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ced15e-3802-4a7e-9db7-3eaa47c68316 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:22.687653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ced15e-3802-4a7e-9db7-3eaa47c68316 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:22.687990 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ced15e-3802-4a7e-9db7-3eaa47c68316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:22.710958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b9ced15e-3802-4a7e-9db7-3eaa47c68316 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:22.712378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b9ced15e-3802-4a7e-9db7-3eaa47c68316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:22.847148 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1319/2627] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:22.856858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367b1fe-a79f-4682-8515-b883f81e2b60 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:22.865255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ced15e-3802-4a7e-9db7-3eaa47c68316 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:22.867968 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b9ced15e-3802-4a7e-9db7-3eaa47c68316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:23.036809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ced15e-3802-4a7e-9db7-3eaa47c68316 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:23.049851 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f08321-7157-4755-94ec-ebcc20b64141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:23.050662 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08321-7157-4755-94ec-ebcc20b64141 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:23.051060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08321-7157-4755-94ec-ebcc20b64141 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:23.051405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08321-7157-4755-94ec-ebcc20b64141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:23.081274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344100d7-07e3-46ad-8cdd-8b3e1c69e3fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:23.082281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344100d7-07e3-46ad-8cdd-8b3e1c69e3fe service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:23.082803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344100d7-07e3-46ad-8cdd-8b3e1c69e3fe service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:23.083271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344100d7-07e3-46ad-8cdd-8b3e1c69e3fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:23.085675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-344100d7-07e3-46ad-8cdd-8b3e1c69e3fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:23.098011 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f08321-7157-4755-94ec-ebcc20b64141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:23.102547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f08321-7157-4755-94ec-ebcc20b64141 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:23.137547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1320/2628] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:23.266510 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344100d7-07e3-46ad-8cdd-8b3e1c69e3fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:23.282189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344100d7-07e3-46ad-8cdd-8b3e1c69e3fe service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:23.340802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1309/2629] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:23.791984 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f08321-7157-4755-94ec-ebcc20b64141 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:23.794005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-57f08321-7157-4755-94ec-ebcc20b64141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:23.816331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f08321-7157-4755-94ec-ebcc20b64141 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:23.828391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:23.829674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:23.829955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:23.830415 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] PATH_INFO: `/v3/projects/d11e9192133c492ab084516adc55f181` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:23.836940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11e9192133c492ab084516adc55f181)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:23.845288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:23.956261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11e9192133c492ab084516adc55f181', 'request_id': 'req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:23.986619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11e9192133c492ab084516adc55f181', 'request_id': 'req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:24.029421 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11e9192133c492ab084516adc55f181', 'request_id': 'req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:24.050579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] The token cache is being invalidate because project d11e9192133c492ab084516adc55f181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:24.051705 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:24.052767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fbf9ed71-4f70-49ad-87a9-a7a0fb36772d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:24.075173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1321/2630] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:23 2022] DELETE /identity/v3/projects/d11e9192133c492ab084516adc55f181 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:24.076925 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344100d7-07e3-46ad-8cdd-8b3e1c69e3fe service nova] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:24.112725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344100d7-07e3-46ad-8cdd-8b3e1c69e3fe service nova] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:24.297995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:24.300123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:24.300749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:24.302097 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] PATH_INFO: `/v3/users/b1f29500ea154582a5e7c028f4beac1a` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:24.304714 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1f29500ea154582a5e7c028f4beac1a)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:24.307424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:24.411223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1f29500ea154582a5e7c028f4beac1a'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:24.413117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1f29500ea154582a5e7c028f4beac1a'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:24.456823 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1f29500ea154582a5e7c028f4beac1a'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:24.458455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1f29500ea154582a5e7c028f4beac1a'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:24.485367 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1f29500ea154582a5e7c028f4beac1a'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:24.550158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1310/2631] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:24 2022] DELETE /identity/v3/users/b1f29500ea154582a5e7c028f4beac1a => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:24.758431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef1c5b-3884-4d95-bd8c-62adab09cccb None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:24.758760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef1c5b-3884-4d95-bd8c-62adab09cccb None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:24.759267 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef1c5b-3884-4d95-bd8c-62adab09cccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:24.804304 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1fef1c5b-3884-4d95-bd8c-62adab09cccb None None] MFA Rules not processed for user `3bbfb51ebb5f4553a9ef7b70becf1a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:24.805713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1fef1c5b-3884-4d95-bd8c-62adab09cccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:24.907497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1322/2632] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:24.913511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:24.915967 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:25.034009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b264b-36b3-46bc-8bd9-01a31abbe4b0 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:25.039315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2598b1-9e5c-4307-b96d-ad7a22df40c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:25.040008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2598b1-9e5c-4307-b96d-ad7a22df40c3 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.040287 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2598b1-9e5c-4307-b96d-ad7a22df40c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.040565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2598b1-9e5c-4307-b96d-ad7a22df40c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.041342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2598b1-9e5c-4307-b96d-ad7a22df40c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:25.043973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2598b1-9e5c-4307-b96d-ad7a22df40c3 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:25.085480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1311/2633] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:24 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:25.095874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef1c5b-3884-4d95-bd8c-62adab09cccb None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:25.099457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2598b1-9e5c-4307-b96d-ad7a22df40c3 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:25.102656 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-4f2598b1-9e5c-4307-b96d-ad7a22df40c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:25.112807 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef1c5b-3884-4d95-bd8c-62adab09cccb None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:25.217870 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2598b1-9e5c-4307-b96d-ad7a22df40c3 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:25.225389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:25.226399 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.226676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.227061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] PATH_INFO: `/v3/projects/95f55d0ed5c4457597132319d6a22d47` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.228879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f55d0ed5c4457597132319d6a22d47)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:25.231795 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:25.235110 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8005be-4b0b-40c4-be21-8acbcaba120a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:25.236115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8005be-4b0b-40c4-be21-8acbcaba120a admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.236405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8005be-4b0b-40c4-be21-8acbcaba120a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.236689 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8005be-4b0b-40c4-be21-8acbcaba120a admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.237424 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8005be-4b0b-40c4-be21-8acbcaba120a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:25.239657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8005be-4b0b-40c4-be21-8acbcaba120a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:25.268459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1323/2634] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:25 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:25.275403 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8005be-4b0b-40c4-be21-8acbcaba120a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:25.293714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8005be-4b0b-40c4-be21-8acbcaba120a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:25.317705 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f55d0ed5c4457597132319d6a22d47', 'request_id': 'req-48a5e1d1-9737-436d-86f2-5dc052965b89'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:25.329578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f55d0ed5c4457597132319d6a22d47', 'request_id': 'req-48a5e1d1-9737-436d-86f2-5dc052965b89'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:25.339545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f55d0ed5c4457597132319d6a22d47', 'request_id': 'req-48a5e1d1-9737-436d-86f2-5dc052965b89'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:25.365722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] The token cache is being invalidate because project 95f55d0ed5c4457597132319d6a22d47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:25.366123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:25.366902 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-48a5e1d1-9737-436d-86f2-5dc052965b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:25.368857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1312/2635] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:25 2022] DELETE /identity/v3/projects/95f55d0ed5c4457597132319d6a22d47 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:25.376479 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d164ae3-b4a6-4a17-b5d4-b7330354c86e None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.377072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d164ae3-b4a6-4a17-b5d4-b7330354c86e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.377621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d164ae3-b4a6-4a17-b5d4-b7330354c86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.391459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739153-b362-4252-9f94-7dddd3424719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:25.392192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739153-b362-4252-9f94-7dddd3424719 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.392520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739153-b362-4252-9f94-7dddd3424719 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.392793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739153-b362-4252-9f94-7dddd3424719 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.393584 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68739153-b362-4252-9f94-7dddd3424719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:25.395291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68739153-b362-4252-9f94-7dddd3424719 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:25.410766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-8d164ae3-b4a6-4a17-b5d4-b7330354c86e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:25.411970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1324/2636] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:25.412279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8d164ae3-b4a6-4a17-b5d4-b7330354c86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:25.412907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739153-b362-4252-9f94-7dddd3424719 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:25.417000 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-68739153-b362-4252-9f94-7dddd3424719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:25.425141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1313/2637] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:25.436005 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04470d6f-3eff-4f63-ab01-b94d4cbec85d None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.436359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739153-b362-4252-9f94-7dddd3424719 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:25.436487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04470d6f-3eff-4f63-ab01-b94d4cbec85d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.437422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04470d6f-3eff-4f63-ab01-b94d4cbec85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.442464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a0a2-f796-46c3-a6f5-f1c3d1ba0f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:25.443418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a0a2-f796-46c3-a6f5-f1c3d1ba0f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.443701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a0a2-f796-46c3-a6f5-f1c3d1ba0f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.443979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a0a2-f796-46c3-a6f5-f1c3d1ba0f55 admin admin] PATH_INFO: `/v3/projects/089f0151b64d4854b0e0391b5aa74c8c/users/ffb5f423e9cf48e2ac65191bb5d5c0b7/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.466507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-04470d6f-3eff-4f63-ab01-b94d4cbec85d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:25.468577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20a0a2-f796-46c3-a6f5-f1c3d1ba0f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=089f0151b64d4854b0e0391b5aa74c8c, user_id=ffb5f423e9cf48e2ac65191bb5d5c0b7, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:25.469424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-04470d6f-3eff-4f63-ab01-b94d4cbec85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:25.472117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20a0a2-f796-46c3-a6f5-f1c3d1ba0f55 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:25.482450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1314/2638] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:25.487888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04470d6f-3eff-4f63-ab01-b94d4cbec85d None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:25.491848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1325/2639] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:25 2022] PUT /identity/v3/projects/089f0151b64d4854b0e0391b5aa74c8c/users/ffb5f423e9cf48e2ac65191bb5d5c0b7/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:25.498251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a0a2-f796-46c3-a6f5-f1c3d1ba0f55 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:25.508074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04470d6f-3eff-4f63-ab01-b94d4cbec85d None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:25.591246 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20a0a2-f796-46c3-a6f5-f1c3d1ba0f55 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:25.597894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf44b9-e04e-4ea2-bb1d-f1a88a82e432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:25.599067 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf44b9-e04e-4ea2-bb1d-f1a88a82e432 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.599357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf44b9-e04e-4ea2-bb1d-f1a88a82e432 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.599638 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf44b9-e04e-4ea2-bb1d-f1a88a82e432 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.600868 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf44b9-e04e-4ea2-bb1d-f1a88a82e432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:25.602564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf44b9-e04e-4ea2-bb1d-f1a88a82e432 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:25.610376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466eec8-4724-48bd-a7dd-d9cad62eec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:25.611379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466eec8-4724-48bd-a7dd-d9cad62eec74 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.611753 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466eec8-4724-48bd-a7dd-d9cad62eec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.612080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466eec8-4724-48bd-a7dd-d9cad62eec74 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.612921 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466eec8-4724-48bd-a7dd-d9cad62eec74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:25.613300 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466eec8-4724-48bd-a7dd-d9cad62eec74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:25.615249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466eec8-4724-48bd-a7dd-d9cad62eec74 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:25.616557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1326/2640] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:25.620768 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf44b9-e04e-4ea2-bb1d-f1a88a82e432 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:25.638534 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1315/2641] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:04:25.642349 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf44b9-e04e-4ea2-bb1d-f1a88a82e432 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:25.644179 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466eec8-4724-48bd-a7dd-d9cad62eec74 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:25.654091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8922c4-271c-4c0e-bc9d-71254faed694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:25.656029 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8922c4-271c-4c0e-bc9d-71254faed694 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.656376 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8922c4-271c-4c0e-bc9d-71254faed694 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.656615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8922c4-271c-4c0e-bc9d-71254faed694 admin admin] PATH_INFO: `/v3/projects/089f0151b64d4854b0e0391b5aa74c8c/users/ffb5f423e9cf48e2ac65191bb5d5c0b7/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.659766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8922c4-271c-4c0e-bc9d-71254faed694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=089f0151b64d4854b0e0391b5aa74c8c, user_id=ffb5f423e9cf48e2ac65191bb5d5c0b7, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:25.662361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8922c4-271c-4c0e-bc9d-71254faed694 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:25.665646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466eec8-4724-48bd-a7dd-d9cad62eec74 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:25.678529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900a6e-8750-44a9-a4d7-ca5cdcb1cf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:25.679954 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900a6e-8750-44a9-a4d7-ca5cdcb1cf1b admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:25.680585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900a6e-8750-44a9-a4d7-ca5cdcb1cf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:25.681166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900a6e-8750-44a9-a4d7-ca5cdcb1cf1b admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:25.681985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22900a6e-8750-44a9-a4d7-ca5cdcb1cf1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:25.684377 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22900a6e-8750-44a9-a4d7-ca5cdcb1cf1b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:25.687865 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1327/2642] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:25 2022] PUT /identity/v3/projects/089f0151b64d4854b0e0391b5aa74c8c/users/ffb5f423e9cf48e2ac65191bb5d5c0b7/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:25.758575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1316/2643] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:25 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:04:25.764129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8922c4-271c-4c0e-bc9d-71254faed694 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:25.880487 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8922c4-271c-4c0e-bc9d-71254faed694 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:25.999197 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b21d45-0f83-48ed-8f48-0eb5f5b71f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:26.000111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b21d45-0f83-48ed-8f48-0eb5f5b71f76 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:26.000111 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b21d45-0f83-48ed-8f48-0eb5f5b71f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:26.000953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b21d45-0f83-48ed-8f48-0eb5f5b71f76 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:26.001899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b21d45-0f83-48ed-8f48-0eb5f5b71f76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:26.005118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b21d45-0f83-48ed-8f48-0eb5f5b71f76 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:26.055720 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1328/2644] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:25 2022] POST /identity/v3/users => generated 288 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:26.060797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900a6e-8750-44a9-a4d7-ca5cdcb1cf1b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:26.102119 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900a6e-8750-44a9-a4d7-ca5cdcb1cf1b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:26.123512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dd406-1a01-4b49-bdb9-dc11e161c0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:26.125288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dd406-1a01-4b49-bdb9-dc11e161c0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:26.125650 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dd406-1a01-4b49-bdb9-dc11e161c0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:26.126032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dd406-1a01-4b49-bdb9-dc11e161c0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:26.126912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67dd406-1a01-4b49-bdb9-dc11e161c0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:26.129943 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67dd406-1a01-4b49-bdb9-dc11e161c0e0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:26.147880 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1317/2645] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:04:26.162262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b21d45-0f83-48ed-8f48-0eb5f5b71f76 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:26.187583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b21d45-0f83-48ed-8f48-0eb5f5b71f76 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:26.204864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fed887-1060-4ad2-9f83-9bcddb6d6821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:26.206010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed887-1060-4ad2-9f83-9bcddb6d6821 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:26.206476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed887-1060-4ad2-9f83-9bcddb6d6821 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:26.206989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed887-1060-4ad2-9f83-9bcddb6d6821 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:26.208346 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fed887-1060-4ad2-9f83-9bcddb6d6821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:26.211314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fed887-1060-4ad2-9f83-9bcddb6d6821 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:26.228723 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1329/2646] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:26.233358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dd406-1a01-4b49-bdb9-dc11e161c0e0 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:26.254749 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dd406-1a01-4b49-bdb9-dc11e161c0e0 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:26.263607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be5681-340e-4a2f-ae9b-4bb9ad9ef683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:26.265936 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be5681-340e-4a2f-ae9b-4bb9ad9ef683 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:26.266676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be5681-340e-4a2f-ae9b-4bb9ad9ef683 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:26.267922 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be5681-340e-4a2f-ae9b-4bb9ad9ef683 admin admin] PATH_INFO: `/v3/projects/8a7084147cb34bdfa8c72a5de3b84ea9/users/b45217b58eab41c1872275a52d0dac2d/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:26.307649 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be5681-340e-4a2f-ae9b-4bb9ad9ef683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7084147cb34bdfa8c72a5de3b84ea9, user_id=b45217b58eab41c1872275a52d0dac2d, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:26.310764 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be5681-340e-4a2f-ae9b-4bb9ad9ef683 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:26.342920 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1318/2647] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:26 2022] PUT /identity/v3/projects/8a7084147cb34bdfa8c72a5de3b84ea9/users/b45217b58eab41c1872275a52d0dac2d/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:26.353173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c4908-4db2-4502-bcec-7f90e3516a54 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:26.353462 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c4908-4db2-4502-bcec-7f90e3516a54 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:26.353742 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c4908-4db2-4502-bcec-7f90e3516a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:26.406610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-5c5c4908-4db2-4502-bcec-7f90e3516a54 None None] MFA Rules not processed for user `b45217b58eab41c1872275a52d0dac2d`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:26.406610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5c5c4908-4db2-4502-bcec-7f90e3516a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:26.594033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1330/2648] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:26.612909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be5681-340e-4a2f-ae9b-4bb9ad9ef683 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:26.615744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e6be5681-340e-4a2f-ae9b-4bb9ad9ef683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:26.775894 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be5681-340e-4a2f-ae9b-4bb9ad9ef683 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:26.791814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19e152-e558-4eae-9c9e-112c5972e62d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4f45e859d84ceaa5a01477e83e098e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:26.792850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19e152-e558-4eae-9c9e-112c5972e62d service glance] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:26.794904 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19e152-e558-4eae-9c9e-112c5972e62d service glance] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:26.795466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19e152-e558-4eae-9c9e-112c5972e62d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:26.832822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19e152-e558-4eae-9c9e-112c5972e62d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:26.835563 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19e152-e558-4eae-9c9e-112c5972e62d service glance] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:26.856886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c4908-4db2-4502-bcec-7f90e3516a54 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:26.856886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5c5c4908-4db2-4502-bcec-7f90e3516a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:26.900429 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1319/2649] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:04:27.054406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c4908-4db2-4502-bcec-7f90e3516a54 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:27.075606 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:27.076747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:27.076933 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:27.077047 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] PATH_INFO: `/v3/users/f9b67061bf654460a19409dc1ec93b76` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:27.078774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9b67061bf654460a19409dc1ec93b76)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:27.081570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:27.192824 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9b67061bf654460a19409dc1ec93b76'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:27.243697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9b67061bf654460a19409dc1ec93b76'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:27.265675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9b67061bf654460a19409dc1ec93b76'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:27.266637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9b67061bf654460a19409dc1ec93b76'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:27.291804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9b67061bf654460a19409dc1ec93b76'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:27.313576 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1331/2650] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:26 2022] DELETE /identity/v3/users/f9b67061bf654460a19409dc1ec93b76 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:27.320217 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aaddb-c392-4d22-b188-7b6efe69b126 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:27.320620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aaddb-c392-4d22-b188-7b6efe69b126 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:27.321736 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aaddb-c392-4d22-b188-7b6efe69b126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:27.366284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-240aaddb-c392-4d22-b188-7b6efe69b126 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:27.368857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-240aaddb-c392-4d22-b188-7b6efe69b126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:27.522863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1320/2651] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:27.536072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:27.538575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:27.694905 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b9bf3-9837-4293-b551-a48bcb7f171b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:27.707137 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55644efb-a132-4cc5-8b40-49ac113bcfef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:27.707986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55644efb-a132-4cc5-8b40-49ac113bcfef service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:27.708272 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55644efb-a132-4cc5-8b40-49ac113bcfef service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:27.708552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55644efb-a132-4cc5-8b40-49ac113bcfef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:27.743250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55644efb-a132-4cc5-8b40-49ac113bcfef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:27.744661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55644efb-a132-4cc5-8b40-49ac113bcfef service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:27.773363 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1332/2652] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:28.431622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aaddb-c392-4d22-b188-7b6efe69b126 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:28.434154 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-240aaddb-c392-4d22-b188-7b6efe69b126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:28.471360 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aaddb-c392-4d22-b188-7b6efe69b126 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:28.483461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:28.485596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:28.486081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:28.486640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] PATH_INFO: `/v3/projects/f6e66a5c52a440f6b64881277a23328c` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:28.488200 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e66a5c52a440f6b64881277a23328c)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:28.505718 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:28.632230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e66a5c52a440f6b64881277a23328c', 'request_id': 'req-eb544439-69ca-42fb-aaa8-fbf181fe490e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:28.645071 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e66a5c52a440f6b64881277a23328c', 'request_id': 'req-eb544439-69ca-42fb-aaa8-fbf181fe490e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:28.660604 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e66a5c52a440f6b64881277a23328c', 'request_id': 'req-eb544439-69ca-42fb-aaa8-fbf181fe490e'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:28.721077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] The token cache is being invalidate because project f6e66a5c52a440f6b64881277a23328c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:28.721445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:28.730096 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-eb544439-69ca-42fb-aaa8-fbf181fe490e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:28.739562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1321/2653] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:28 2022] DELETE /identity/v3/projects/f6e66a5c52a440f6b64881277a23328c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:28.745343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788c5d7-6a1b-49b7-af46-afb18c4a917d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:28.746756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788c5d7-6a1b-49b7-af46-afb18c4a917d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:28.747581 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788c5d7-6a1b-49b7-af46-afb18c4a917d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:28.811434 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0788c5d7-6a1b-49b7-af46-afb18c4a917d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:28.812820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0788c5d7-6a1b-49b7-af46-afb18c4a917d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:28.955962 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1333/2654] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:28.966228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847cf15-0f7c-41ce-8f66-e5d53f11e4e5 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:28.966758 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847cf15-0f7c-41ce-8f66-e5d53f11e4e5 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:28.967278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847cf15-0f7c-41ce-8f66-e5d53f11e4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:29.093283 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-5847cf15-0f7c-41ce-8f66-e5d53f11e4e5 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:29.095351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5847cf15-0f7c-41ce-8f66-e5d53f11e4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:29.129929 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1322/2655] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:29.136479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788c5d7-6a1b-49b7-af46-afb18c4a917d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:29.178612 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788c5d7-6a1b-49b7-af46-afb18c4a917d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:29.194973 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b42cdc-f38e-4a1e-97ed-ced2996a3845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:29.196296 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b42cdc-f38e-4a1e-97ed-ced2996a3845 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:29.196583 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b42cdc-f38e-4a1e-97ed-ced2996a3845 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:29.196865 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b42cdc-f38e-4a1e-97ed-ced2996a3845 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:29.198165 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b42cdc-f38e-4a1e-97ed-ced2996a3845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:29.198165 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b42cdc-f38e-4a1e-97ed-ced2996a3845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:29.200595 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b42cdc-f38e-4a1e-97ed-ced2996a3845 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:29.233319 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1334/2656] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:04:29.235900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847cf15-0f7c-41ce-8f66-e5d53f11e4e5 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:29.268820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847cf15-0f7c-41ce-8f66-e5d53f11e4e5 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:29.284115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0460dcf-a747-424f-b8fd-a6b98402e985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:29.285404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0460dcf-a747-424f-b8fd-a6b98402e985 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:29.286160 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0460dcf-a747-424f-b8fd-a6b98402e985 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:29.286824 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0460dcf-a747-424f-b8fd-a6b98402e985 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:29.289834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0460dcf-a747-424f-b8fd-a6b98402e985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:29.295449 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0460dcf-a747-424f-b8fd-a6b98402e985 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:29.368009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1323/2657] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:29 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:29.376280 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b42cdc-f38e-4a1e-97ed-ced2996a3845 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:29.404525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b42cdc-f38e-4a1e-97ed-ced2996a3845 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:29.587010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f56403-a8bd-4530-9fa3-1f17effa8d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:29.587472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f56403-a8bd-4530-9fa3-1f17effa8d9c admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:29.587873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f56403-a8bd-4530-9fa3-1f17effa8d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:29.588446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f56403-a8bd-4530-9fa3-1f17effa8d9c admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:29.589845 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f56403-a8bd-4530-9fa3-1f17effa8d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:29.592816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f56403-a8bd-4530-9fa3-1f17effa8d9c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:29.642080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1335/2658] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:29 2022] POST /identity/v3/users => generated 283 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:29.648156 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0460dcf-a747-424f-b8fd-a6b98402e985 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:29.666756 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0460dcf-a747-424f-b8fd-a6b98402e985 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:29.685484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23376337-eb05-4245-8db9-35d5d54fea9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:29.686175 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23376337-eb05-4245-8db9-35d5d54fea9b admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:29.686454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23376337-eb05-4245-8db9-35d5d54fea9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:29.686910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23376337-eb05-4245-8db9-35d5d54fea9b admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:29.687750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23376337-eb05-4245-8db9-35d5d54fea9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:29.689619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23376337-eb05-4245-8db9-35d5d54fea9b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:29.710662 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1324/2659] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:29.716038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f56403-a8bd-4530-9fa3-1f17effa8d9c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:29.732886 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f56403-a8bd-4530-9fa3-1f17effa8d9c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:29.740981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf116bd-cb9f-4613-b07f-5b8e7cf53aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:29.741707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf116bd-cb9f-4613-b07f-5b8e7cf53aaa admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:29.742089 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf116bd-cb9f-4613-b07f-5b8e7cf53aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:29.742406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf116bd-cb9f-4613-b07f-5b8e7cf53aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:29.743408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf116bd-cb9f-4613-b07f-5b8e7cf53aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:29.745668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf116bd-cb9f-4613-b07f-5b8e7cf53aaa admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:29.761988 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1336/2660] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:29.768421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23376337-eb05-4245-8db9-35d5d54fea9b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:29.795378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23376337-eb05-4245-8db9-35d5d54fea9b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:29.806014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023f037-9c7e-4c57-b8b7-a17a4493fb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:29.808049 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023f037-9c7e-4c57-b8b7-a17a4493fb6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:29.808541 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023f037-9c7e-4c57-b8b7-a17a4493fb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:29.809275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023f037-9c7e-4c57-b8b7-a17a4493fb6b admin admin] PATH_INFO: `/v3/projects/3dccf71e49b74967b8a7709ef00823cb/users/a8fe3a3e02d64d0db21d1cec3fcaca0b/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:29.846448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023f037-9c7e-4c57-b8b7-a17a4493fb6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dccf71e49b74967b8a7709ef00823cb, user_id=a8fe3a3e02d64d0db21d1cec3fcaca0b, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:29.850025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023f037-9c7e-4c57-b8b7-a17a4493fb6b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:29.871038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1325/2661] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:29 2022] PUT /identity/v3/projects/3dccf71e49b74967b8a7709ef00823cb/users/a8fe3a3e02d64d0db21d1cec3fcaca0b/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:29.890663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7d58a-04b5-4dfd-bcb8-0ee353aad0e6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:29.890983 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7d58a-04b5-4dfd-bcb8-0ee353aad0e6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:29.892289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7d58a-04b5-4dfd-bcb8-0ee353aad0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:29.945264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a0f7d58a-04b5-4dfd-bcb8-0ee353aad0e6 None None] MFA Rules not processed for user `a8fe3a3e02d64d0db21d1cec3fcaca0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:29.946799 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a0f7d58a-04b5-4dfd-bcb8-0ee353aad0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:30.149809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1337/2662] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:30.150535 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023f037-9c7e-4c57-b8b7-a17a4493fb6b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:30.297915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023f037-9c7e-4c57-b8b7-a17a4493fb6b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:30.306174 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad2ced-d77f-4171-a13e-f56b0b7edfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:30.307447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad2ced-d77f-4171-a13e-f56b0b7edfce admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:30.307862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad2ced-d77f-4171-a13e-f56b0b7edfce admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:30.308282 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad2ced-d77f-4171-a13e-f56b0b7edfce admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:30.309497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad2ced-d77f-4171-a13e-f56b0b7edfce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:30.313820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad2ced-d77f-4171-a13e-f56b0b7edfce admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:30.389540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1326/2663] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:30 2022] POST /identity/v3/projects => generated 363 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:04:30.391590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7d58a-04b5-4dfd-bcb8-0ee353aad0e6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:30.423374 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7d58a-04b5-4dfd-bcb8-0ee353aad0e6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:30.574515 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2afdf-34a0-490a-a87e-47ca93c7f5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:30.575643 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2afdf-34a0-490a-a87e-47ca93c7f5c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:30.576913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2afdf-34a0-490a-a87e-47ca93c7f5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:30.577337 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2afdf-34a0-490a-a87e-47ca93c7f5c9 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:30.578744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2afdf-34a0-490a-a87e-47ca93c7f5c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:30.582888 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2afdf-34a0-490a-a87e-47ca93c7f5c9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:30.638758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1338/2664] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:30 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:30.646011 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad2ced-d77f-4171-a13e-f56b0b7edfce admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:30.678084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad2ced-d77f-4171-a13e-f56b0b7edfce admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:30.686490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936bb40-4f57-44f5-aaad-242d5afbe418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:30.688100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936bb40-4f57-44f5-aaad-242d5afbe418 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:30.688526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936bb40-4f57-44f5-aaad-242d5afbe418 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:30.688987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936bb40-4f57-44f5-aaad-242d5afbe418 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:30.693192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936bb40-4f57-44f5-aaad-242d5afbe418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:30.694062 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936bb40-4f57-44f5-aaad-242d5afbe418 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:30.729460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1327/2665] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:30.736359 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2afdf-34a0-490a-a87e-47ca93c7f5c9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:30.769343 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2afdf-34a0-490a-a87e-47ca93c7f5c9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:30.784199 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99c47f-9f1a-4a5a-94ee-7af931b84882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:30.786687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99c47f-9f1a-4a5a-94ee-7af931b84882 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:30.787385 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99c47f-9f1a-4a5a-94ee-7af931b84882 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:30.788135 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99c47f-9f1a-4a5a-94ee-7af931b84882 admin admin] PATH_INFO: `/v3/projects/e389eb1017c74b109d0d122e21d69e12/users/835f55a749a54e069d4a9ff53f527bbc/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:30.829867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99c47f-9f1a-4a5a-94ee-7af931b84882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e389eb1017c74b109d0d122e21d69e12, user_id=835f55a749a54e069d4a9ff53f527bbc, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:30.838585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99c47f-9f1a-4a5a-94ee-7af931b84882 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:30.873602 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1339/2666] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:30 2022] PUT /identity/v3/projects/e389eb1017c74b109d0d122e21d69e12/users/835f55a749a54e069d4a9ff53f527bbc/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:30.879522 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936bb40-4f57-44f5-aaad-242d5afbe418 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:30.935926 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936bb40-4f57-44f5-aaad-242d5afbe418 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:31.123722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eea2c6-0ec5-41d9-974f-af89d121171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:31.125633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eea2c6-0ec5-41d9-974f-af89d121171b admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:31.126456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eea2c6-0ec5-41d9-974f-af89d121171b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:31.127401 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eea2c6-0ec5-41d9-974f-af89d121171b admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:31.128519 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eea2c6-0ec5-41d9-974f-af89d121171b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:31.130482 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eea2c6-0ec5-41d9-974f-af89d121171b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:31.190812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1328/2667] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:04:31.196956 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99c47f-9f1a-4a5a-94ee-7af931b84882 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:31.216811 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99c47f-9f1a-4a5a-94ee-7af931b84882 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:31.235782 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c9d87-22a1-49ef-9998-bde640d98110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:31.240763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c9d87-22a1-49ef-9998-bde640d98110 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:31.242982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c9d87-22a1-49ef-9998-bde640d98110 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:31.243166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c9d87-22a1-49ef-9998-bde640d98110 admin admin] PATH_INFO: `/v3/projects/e389eb1017c74b109d0d122e21d69e12/users/835f55a749a54e069d4a9ff53f527bbc/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:31.254576 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c9d87-22a1-49ef-9998-bde640d98110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e389eb1017c74b109d0d122e21d69e12, user_id=835f55a749a54e069d4a9ff53f527bbc, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:31.259741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c9d87-22a1-49ef-9998-bde640d98110 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:31.293466 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1340/2668] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:31 2022] PUT /identity/v3/projects/e389eb1017c74b109d0d122e21d69e12/users/835f55a749a54e069d4a9ff53f527bbc/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:31.322213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c82a-ce8f-4dfb-adc8-f1811518639c None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:31.322588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c82a-ce8f-4dfb-adc8-f1811518639c None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:31.323101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c82a-ce8f-4dfb-adc8-f1811518639c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:31.402180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-5a91c82a-ce8f-4dfb-adc8-f1811518639c None None] MFA Rules not processed for user `835f55a749a54e069d4a9ff53f527bbc`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:31.404704 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5a91c82a-ce8f-4dfb-adc8-f1811518639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:31.595491 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1329/2669] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:31 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:31.613914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c9d87-22a1-49ef-9998-bde640d98110 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:31.618627 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9c6c9d87-22a1-49ef-9998-bde640d98110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:31.777522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c9d87-22a1-49ef-9998-bde640d98110 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:31.793327 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036aaeb-28d4-4937-aea1-24d8be7860da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:31.794457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036aaeb-28d4-4937-aea1-24d8be7860da service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:31.795004 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036aaeb-28d4-4937-aea1-24d8be7860da service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:31.796033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036aaeb-28d4-4937-aea1-24d8be7860da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:31.815921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036aaeb-28d4-4937-aea1-24d8be7860da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:31.817628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036aaeb-28d4-4937-aea1-24d8be7860da service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:31.844173 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1341/2670] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:32.813425 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91c82a-ce8f-4dfb-adc8-f1811518639c None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:33.019075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91c82a-ce8f-4dfb-adc8-f1811518639c None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:33.026617 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:33.027804 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:33.028376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:33.028814 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] PATH_INFO: `/v3/users/a8fe3a3e02d64d0db21d1cec3fcaca0b` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:33.035254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8fe3a3e02d64d0db21d1cec3fcaca0b)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:33.040884 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:33.177191 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8fe3a3e02d64d0db21d1cec3fcaca0b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:33.178018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8fe3a3e02d64d0db21d1cec3fcaca0b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:33.200000 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8fe3a3e02d64d0db21d1cec3fcaca0b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:33.201129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8fe3a3e02d64d0db21d1cec3fcaca0b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:33.219610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8fe3a3e02d64d0db21d1cec3fcaca0b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:33.269043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1330/2671] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:32 2022] DELETE /identity/v3/users/a8fe3a3e02d64d0db21d1cec3fcaca0b => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:33.274576 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22343a-535a-4373-98d5-66e252ea650e None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:33.275294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22343a-535a-4373-98d5-66e252ea650e None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:33.275294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22343a-535a-4373-98d5-66e252ea650e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:33.320703 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-5b22343a-535a-4373-98d5-66e252ea650e None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:33.323509 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5b22343a-535a-4373-98d5-66e252ea650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:33.441870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1342/2672] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:33.453242 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:33.455697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:33.569941 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508610a6-1355-4591-b882-2c827c43b247 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:33.581578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b30b9d-aa26-4657-bb24-dab40195fd8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:33.583527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b30b9d-aa26-4657-bb24-dab40195fd8a service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:33.584025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b30b9d-aa26-4657-bb24-dab40195fd8a service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:33.584523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b30b9d-aa26-4657-bb24-dab40195fd8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:33.606271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b30b9d-aa26-4657-bb24-dab40195fd8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:33.607091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b30b9d-aa26-4657-bb24-dab40195fd8a service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:33.620387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1331/2673] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:34.091672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22343a-535a-4373-98d5-66e252ea650e None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:34.094619 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5b22343a-535a-4373-98d5-66e252ea650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:34.137238 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0848e8e-5c80-4de6-8245-ea112df6d0fc None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:34.138039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0848e8e-5c80-4de6-8245-ea112df6d0fc None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:34.138039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0848e8e-5c80-4de6-8245-ea112df6d0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:34.142361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22343a-535a-4373-98d5-66e252ea650e None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:34.151503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:34.153971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:34.156604 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:34.157394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] PATH_INFO: `/v3/projects/3dccf71e49b74967b8a7709ef00823cb` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:34.161061 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dccf71e49b74967b8a7709ef00823cb)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:34.166699 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:34.182665 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-b0848e8e-5c80-4de6-8245-ea112df6d0fc None None] MFA Rules not processed for user `ffb5f423e9cf48e2ac65191bb5d5c0b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:34.188572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b0848e8e-5c80-4de6-8245-ea112df6d0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:34.246438 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dccf71e49b74967b8a7709ef00823cb', 'request_id': 'req-21a7fe79-2d05-4b94-afb7-217fb41b573e'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:34.254541 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dccf71e49b74967b8a7709ef00823cb', 'request_id': 'req-21a7fe79-2d05-4b94-afb7-217fb41b573e'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:34.268728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dccf71e49b74967b8a7709ef00823cb', 'request_id': 'req-21a7fe79-2d05-4b94-afb7-217fb41b573e'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:34.288833 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] The token cache is being invalidate because project 3dccf71e49b74967b8a7709ef00823cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:34.289913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:34.290063 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:34.294639 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1343/2674] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:34 2022] DELETE /identity/v3/projects/3dccf71e49b74967b8a7709ef00823cb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:34.301868 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:34.305923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:34.316915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1332/2675] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:34.332195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0848e8e-5c80-4de6-8245-ea112df6d0fc None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:34.334038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-b0848e8e-5c80-4de6-8245-ea112df6d0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:34.434041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0848e8e-5c80-4de6-8245-ea112df6d0fc None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:34.442477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7fe79-2d05-4b94-afb7-217fb41b573e admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:34.447777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:34.448821 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:34.449098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:34.449674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] PATH_INFO: `/v3/users/835f55a749a54e069d4a9ff53f527bbc` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:34.451096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=835f55a749a54e069d4a9ff53f527bbc)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:34.453114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:34.524256 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '835f55a749a54e069d4a9ff53f527bbc'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:34.533241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebefe58-a82f-497b-aa8c-3a8b54dedb06 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc4f45e859d84ceaa5a01477e83e098e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:34.535093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefe58-a82f-497b-aa8c-3a8b54dedb06 service glance] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:34.535726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefe58-a82f-497b-aa8c-3a8b54dedb06 service glance] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:34.536848 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebefe58-a82f-497b-aa8c-3a8b54dedb06 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:34.541524 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '835f55a749a54e069d4a9ff53f527bbc'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:34.545457 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9ebefe58-a82f-497b-aa8c-3a8b54dedb06 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:34.552186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '835f55a749a54e069d4a9ff53f527bbc'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:34.552955 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '835f55a749a54e069d4a9ff53f527bbc'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:34.568691 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e50761d6-c331-49ce-876d-1e87928249a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '835f55a749a54e069d4a9ff53f527bbc'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:34.591038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1344/2676] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:34 2022] DELETE /identity/v3/users/835f55a749a54e069d4a9ff53f527bbc => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:34.710639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebefe58-a82f-497b-aa8c-3a8b54dedb06 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:34.713638 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebefe58-a82f-497b-aa8c-3a8b54dedb06 service glance] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:34.731744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1333/2677] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:34.899521 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ad34d-feb1-47e2-8555-d69bfca94d17 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:34.900775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ad34d-feb1-47e2-8555-d69bfca94d17 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:34.901603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ad34d-feb1-47e2-8555-d69bfca94d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:34.916731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebefe58-a82f-497b-aa8c-3a8b54dedb06 service glance] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:34.919319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9ebefe58-a82f-497b-aa8c-3a8b54dedb06 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:34.943290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebefe58-a82f-497b-aa8c-3a8b54dedb06 service glance] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:34.945263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-094ad34d-feb1-47e2-8555-d69bfca94d17 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:34.946597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-094ad34d-feb1-47e2-8555-d69bfca94d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:35.035618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:35.036789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:35.037228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:35.037679 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] PATH_INFO: `/v3/users/7862bb758fc7492f866c1cae8d539eee` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:35.039815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7862bb758fc7492f866c1cae8d539eee)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:35.047033 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:35.065132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1345/2678] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 14:04:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:35.132819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ad34d-feb1-47e2-8555-d69bfca94d17 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:35.136740 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-094ad34d-feb1-47e2-8555-d69bfca94d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:35.142402 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7862bb758fc7492f866c1cae8d539eee'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:35.144362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7862bb758fc7492f866c1cae8d539eee'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:35.164431 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ad34d-feb1-47e2-8555-d69bfca94d17 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:35.172603 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:35.173503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:35.173793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:35.174069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] PATH_INFO: `/v3/projects/e389eb1017c74b109d0d122e21d69e12` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:35.175591 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e389eb1017c74b109d0d122e21d69e12)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:35.178135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7862bb758fc7492f866c1cae8d539eee'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:35.179018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:35.179469 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7862bb758fc7492f866c1cae8d539eee'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:35.190025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7862bb758fc7492f866c1cae8d539eee'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:35.228219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1334/2679] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:34 2022] DELETE /identity/v3/users/7862bb758fc7492f866c1cae8d539eee => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:35.228219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:35.228219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:35.269760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e389eb1017c74b109d0d122e21d69e12', 'request_id': 'req-afd0684b-3b21-4770-810c-66ce5fca66d1'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:35.287549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e389eb1017c74b109d0d122e21d69e12', 'request_id': 'req-afd0684b-3b21-4770-810c-66ce5fca66d1'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:35.310144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e389eb1017c74b109d0d122e21d69e12', 'request_id': 'req-afd0684b-3b21-4770-810c-66ce5fca66d1'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:35.339366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] The token cache is being invalidate because project e389eb1017c74b109d0d122e21d69e12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:35.339769 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:35.347428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-afd0684b-3b21-4770-810c-66ce5fca66d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:35.352743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1346/2680] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:35 2022] DELETE /identity/v3/projects/e389eb1017c74b109d0d122e21d69e12 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:35.355387 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220ca8c-25f7-4e29-be1b-aa9d28313a53 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:35.355891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220ca8c-25f7-4e29-be1b-aa9d28313a53 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:35.356381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220ca8c-25f7-4e29-be1b-aa9d28313a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:35.381790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1987b-ddd7-430d-be62-11e74e0a3976 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:35.396476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-9220ca8c-25f7-4e29-be1b-aa9d28313a53 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:35.397834 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9220ca8c-25f7-4e29-be1b-aa9d28313a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:35.498946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1347/2681] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:35.501086 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e620553-833e-4af8-9126-f80938bdd969 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:35.501560 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e620553-833e-4af8-9126-f80938bdd969 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:35.502274 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e620553-833e-4af8-9126-f80938bdd969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:35.526805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da9dea-7fb1-467b-b4dc-57654df9cd4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:35.527165 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da9dea-7fb1-467b-b4dc-57654df9cd4c service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:35.527337 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da9dea-7fb1-467b-b4dc-57654df9cd4c service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:35.527917 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da9dea-7fb1-467b-b4dc-57654df9cd4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:35.530292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-51da9dea-7fb1-467b-b4dc-57654df9cd4c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:35.542581 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-7e620553-833e-4af8-9126-f80938bdd969 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:35.544050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7e620553-833e-4af8-9126-f80938bdd969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:35.555545 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1348/2682] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:35.557371 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e620553-833e-4af8-9126-f80938bdd969 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:35.565864 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7e620553-833e-4af8-9126-f80938bdd969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:35.633812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da9dea-7fb1-467b-b4dc-57654df9cd4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:35.637043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da9dea-7fb1-467b-b4dc-57654df9cd4c service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:35.647364 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1335/2683] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:04:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 14:04:35.648883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce7a7-23f8-43b7-8ff9-63c26fb619cb None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:35.649170 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce7a7-23f8-43b7-8ff9-63c26fb619cb None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:35.649471 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce7a7-23f8-43b7-8ff9-63c26fb619cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:35.664166 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e620553-833e-4af8-9126-f80938bdd969 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:35.669082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-e37ce7a7-23f8-43b7-8ff9-63c26fb619cb None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:35.670397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e37ce7a7-23f8-43b7-8ff9-63c26fb619cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:35.671935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1efa4d-fa1e-4752-9099-b3fb6c50bca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:35.673228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1efa4d-fa1e-4752-9099-b3fb6c50bca1 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:35.674034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1efa4d-fa1e-4752-9099-b3fb6c50bca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:35.674260 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1efa4d-fa1e-4752-9099-b3fb6c50bca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:35.698767 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1efa4d-fa1e-4752-9099-b3fb6c50bca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:35.701018 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1efa4d-fa1e-4752-9099-b3fb6c50bca1 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:35.705023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1336/2684] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:35.711100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce7a7-23f8-43b7-8ff9-63c26fb619cb None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:35.730596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce7a7-23f8-43b7-8ff9-63c26fb619cb None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:35.732183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1349/2685] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:35.746620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358d3a2-0de8-4eb5-959b-8e481b6c6d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:35.747984 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358d3a2-0de8-4eb5-959b-8e481b6c6d7b admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:35.748443 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358d3a2-0de8-4eb5-959b-8e481b6c6d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:35.748901 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358d3a2-0de8-4eb5-959b-8e481b6c6d7b admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:35.751843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358d3a2-0de8-4eb5-959b-8e481b6c6d7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:35.752345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358d3a2-0de8-4eb5-959b-8e481b6c6d7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:35.756595 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358d3a2-0de8-4eb5-959b-8e481b6c6d7b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:35.796068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1337/2686] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:04:35.800764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1efa4d-fa1e-4752-9099-b3fb6c50bca1 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:35.822348 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1efa4d-fa1e-4752-9099-b3fb6c50bca1 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:35.836587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe768c-a6ad-43f1-9e99-0147dedc8bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:35.838758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe768c-a6ad-43f1-9e99-0147dedc8bba admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:35.838758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe768c-a6ad-43f1-9e99-0147dedc8bba admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:35.838758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe768c-a6ad-43f1-9e99-0147dedc8bba admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:35.839820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe768c-a6ad-43f1-9e99-0147dedc8bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:35.842732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe768c-a6ad-43f1-9e99-0147dedc8bba admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:35.945970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1350/2687] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:35 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:35.952498 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358d3a2-0de8-4eb5-959b-8e481b6c6d7b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:35.977166 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358d3a2-0de8-4eb5-959b-8e481b6c6d7b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:36.161266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9da2d-75bc-49a6-b1a4-f1c1d9f0b7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:36.162999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9da2d-75bc-49a6-b1a4-f1c1d9f0b7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:36.163629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9da2d-75bc-49a6-b1a4-f1c1d9f0b7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:36.164219 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9da2d-75bc-49a6-b1a4-f1c1d9f0b7e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:36.165936 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9da2d-75bc-49a6-b1a4-f1c1d9f0b7e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:36.168195 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9da2d-75bc-49a6-b1a4-f1c1d9f0b7e2 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:36.214743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe768c-a6ad-43f1-9e99-0147dedc8bba admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:36.215387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1338/2688] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:35 2022] POST /identity/v3/users => generated 285 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:36.262497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe768c-a6ad-43f1-9e99-0147dedc8bba admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:36.302128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d55e8a-8547-4647-99c7-f27ab418b1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:36.305039 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d55e8a-8547-4647-99c7-f27ab418b1dd admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:36.306648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d55e8a-8547-4647-99c7-f27ab418b1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:36.309277 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d55e8a-8547-4647-99c7-f27ab418b1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:36.313273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d55e8a-8547-4647-99c7-f27ab418b1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:36.319376 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d55e8a-8547-4647-99c7-f27ab418b1dd admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:36.354602 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9da2d-75bc-49a6-b1a4-f1c1d9f0b7e2 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:36.361751 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ddb9da2d-75bc-49a6-b1a4-f1c1d9f0b7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:36.372252 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1351/2689] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:36.385153 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d55e8a-8547-4647-99c7-f27ab418b1dd admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:36.400017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9da2d-75bc-49a6-b1a4-f1c1d9f0b7e2 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:36.415777 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d55e8a-8547-4647-99c7-f27ab418b1dd admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:36.422869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:36.423797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:36.424818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:36.424818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] PATH_INFO: `/v3/projects/37646edfd30f4131b251f5c649b423bc` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:36.427986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37646edfd30f4131b251f5c649b423bc)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:36.430842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:36.436367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be2e3a-b43e-4e3a-96fc-f43e253ca0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:36.438143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be2e3a-b43e-4e3a-96fc-f43e253ca0da admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:36.438895 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be2e3a-b43e-4e3a-96fc-f43e253ca0da admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:36.439707 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be2e3a-b43e-4e3a-96fc-f43e253ca0da admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:36.441126 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be2e3a-b43e-4e3a-96fc-f43e253ca0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:36.447058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be2e3a-b43e-4e3a-96fc-f43e253ca0da admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:36.481106 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1352/2690] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:36.491748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be2e3a-b43e-4e3a-96fc-f43e253ca0da admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:36.543334 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be2e3a-b43e-4e3a-96fc-f43e253ca0da admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:36.619596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37646edfd30f4131b251f5c649b423bc', 'request_id': 'req-d442f9d5-2062-4288-9302-49c979dcff30'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:36.641612 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37646edfd30f4131b251f5c649b423bc', 'request_id': 'req-d442f9d5-2062-4288-9302-49c979dcff30'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:36.659513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37646edfd30f4131b251f5c649b423bc', 'request_id': 'req-d442f9d5-2062-4288-9302-49c979dcff30'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:36.695467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] The token cache is being invalidate because project 37646edfd30f4131b251f5c649b423bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:36.695958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:36.697332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-d442f9d5-2062-4288-9302-49c979dcff30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:36.700341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1339/2691] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:36 2022] DELETE /identity/v3/projects/37646edfd30f4131b251f5c649b423bc => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:36.708366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8eb06-0650-405f-8126-0a162a594ade None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:36.708490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8eb06-0650-405f-8126-0a162a594ade None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:36.708755 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8eb06-0650-405f-8126-0a162a594ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:36.743229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7cf8eb06-0650-405f-8126-0a162a594ade None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:36.749555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7cf8eb06-0650-405f-8126-0a162a594ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:36.760388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c0314-75f5-44f8-a123-d8841a5ad2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:36.761520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c0314-75f5-44f8-a123-d8841a5ad2bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:36.762452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c0314-75f5-44f8-a123-d8841a5ad2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:36.763695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c0314-75f5-44f8-a123-d8841a5ad2bd admin admin] PATH_INFO: `/v3/projects/06882817e9574bd48f428c200dceb794/users/08e5b316aeb5484890ac78972e19221f/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:36.775259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1340/2692] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:36.833663 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c0314-75f5-44f8-a123-d8841a5ad2bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=06882817e9574bd48f428c200dceb794, user_id=08e5b316aeb5484890ac78972e19221f, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:36.847685 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c0314-75f5-44f8-a123-d8841a5ad2bd admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:36.879045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a0d08-fe9d-4909-9457-e8eaf6a517c7 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:36.879426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a0d08-fe9d-4909-9457-e8eaf6a517c7 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:36.879670 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a0d08-fe9d-4909-9457-e8eaf6a517c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:36.902789 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1353/2693] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:36 2022] PUT /identity/v3/projects/06882817e9574bd48f428c200dceb794/users/08e5b316aeb5484890ac78972e19221f/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:36.907606 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59880b1-5a5e-46b2-8ab9-5ae60a6f125d None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:36.908552 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59880b1-5a5e-46b2-8ab9-5ae60a6f125d None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:36.908848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59880b1-5a5e-46b2-8ab9-5ae60a6f125d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:36.918657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-ef7a0d08-fe9d-4909-9457-e8eaf6a517c7 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:36.920666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-ef7a0d08-fe9d-4909-9457-e8eaf6a517c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:36.982899 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a59880b1-5a5e-46b2-8ab9-5ae60a6f125d None None] MFA Rules not processed for user `08e5b316aeb5484890ac78972e19221f`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:36.985446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a59880b1-5a5e-46b2-8ab9-5ae60a6f125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:37.073793 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1341/2694] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:37.075611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a0d08-fe9d-4909-9457-e8eaf6a517c7 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.107252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a0d08-fe9d-4909-9457-e8eaf6a517c7 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.115580 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd06c52-76c7-414f-b965-0b6840db9958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.117259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd06c52-76c7-414f-b965-0b6840db9958 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.117955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd06c52-76c7-414f-b965-0b6840db9958 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.118390 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd06c52-76c7-414f-b965-0b6840db9958 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.119209 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd06c52-76c7-414f-b965-0b6840db9958 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:37.119512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd06c52-76c7-414f-b965-0b6840db9958 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.121844 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd06c52-76c7-414f-b965-0b6840db9958 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.172979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1342/2695] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:04:37.177338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd06c52-76c7-414f-b965-0b6840db9958 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.192504 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd06c52-76c7-414f-b965-0b6840db9958 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.205551 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a1efe-f8bf-4dd8-9d5f-090221a79668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.206694 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a1efe-f8bf-4dd8-9d5f-090221a79668 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.208124 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a1efe-f8bf-4dd8-9d5f-090221a79668 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.208594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a1efe-f8bf-4dd8-9d5f-090221a79668 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.209722 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a1efe-f8bf-4dd8-9d5f-090221a79668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.213669 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a1efe-f8bf-4dd8-9d5f-090221a79668 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.229894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1354/2696] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:37.231741 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59880b1-5a5e-46b2-8ab9-5ae60a6f125d None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.234339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a59880b1-5a5e-46b2-8ab9-5ae60a6f125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:37.264656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59880b1-5a5e-46b2-8ab9-5ae60a6f125d None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.270251 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ddb2b-30e0-4881-9239-dd61c835bae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.270958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ddb2b-30e0-4881-9239-dd61c835bae9 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.271241 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ddb2b-30e0-4881-9239-dd61c835bae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.271513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ddb2b-30e0-4881-9239-dd61c835bae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.272215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ddb2b-30e0-4881-9239-dd61c835bae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.276571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1343/2697] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:37 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:37.276763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ddb2b-30e0-4881-9239-dd61c835bae9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.278015 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a1efe-f8bf-4dd8-9d5f-090221a79668 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.338072 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1355/2698] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:37 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:37.345539 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ddb2b-30e0-4881-9239-dd61c835bae9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.412438 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a1efe-f8bf-4dd8-9d5f-090221a79668 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.456606 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ddb2b-30e0-4881-9239-dd61c835bae9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.462839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55060c89-3f49-49db-825e-a92bfcc2ee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.463345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55060c89-3f49-49db-825e-a92bfcc2ee2a admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.463635 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55060c89-3f49-49db-825e-a92bfcc2ee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.463944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55060c89-3f49-49db-825e-a92bfcc2ee2a admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.470919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55060c89-3f49-49db-825e-a92bfcc2ee2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.473289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55060c89-3f49-49db-825e-a92bfcc2ee2a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.511175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1356/2699] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:37 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:37.512948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55060c89-3f49-49db-825e-a92bfcc2ee2a admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.527311 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e154e6d-e3d1-4e99-8d7a-3eaec989ae80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.527996 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e154e6d-e3d1-4e99-8d7a-3eaec989ae80 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.528275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e154e6d-e3d1-4e99-8d7a-3eaec989ae80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.528549 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e154e6d-e3d1-4e99-8d7a-3eaec989ae80 admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.529237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e154e6d-e3d1-4e99-8d7a-3eaec989ae80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.531346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e154e6d-e3d1-4e99-8d7a-3eaec989ae80 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.535735 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55060c89-3f49-49db-825e-a92bfcc2ee2a admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.544012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f81aad-3d33-4191-abd0-4a0b65f39e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.545405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f81aad-3d33-4191-abd0-4a0b65f39e0f admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.546220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f81aad-3d33-4191-abd0-4a0b65f39e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.548219 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f81aad-3d33-4191-abd0-4a0b65f39e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.549882 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f81aad-3d33-4191-abd0-4a0b65f39e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.551765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f81aad-3d33-4191-abd0-4a0b65f39e0f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.567134 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1344/2700] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:37 2022] POST /identity/v3/users => generated 286 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:37.567342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1357/2701] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:37.571512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f81aad-3d33-4191-abd0-4a0b65f39e0f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.572136 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e154e6d-e3d1-4e99-8d7a-3eaec989ae80 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.588567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e154e6d-e3d1-4e99-8d7a-3eaec989ae80 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.590730 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f81aad-3d33-4191-abd0-4a0b65f39e0f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.595382 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d7ecc-99bf-40cc-b59a-f6ae2026dc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.596482 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d7ecc-99bf-40cc-b59a-f6ae2026dc83 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.596840 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d7ecc-99bf-40cc-b59a-f6ae2026dc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.596981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c024b12-0bd4-4a9c-a261-76456ae6ca2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.597259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d7ecc-99bf-40cc-b59a-f6ae2026dc83 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.598064 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d7ecc-99bf-40cc-b59a-f6ae2026dc83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.598322 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c024b12-0bd4-4a9c-a261-76456ae6ca2f admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.598878 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c024b12-0bd4-4a9c-a261-76456ae6ca2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.599382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c024b12-0bd4-4a9c-a261-76456ae6ca2f admin admin] PATH_INFO: `/v3/projects/64118111a1994c3b84aa9d8d8126fddf/users/566558040bc34b1bb145df1c76c7a51d/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.600218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d7ecc-99bf-40cc-b59a-f6ae2026dc83 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.616418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1345/2702] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:04:37.616418 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d7ecc-99bf-40cc-b59a-f6ae2026dc83 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.630017 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d7ecc-99bf-40cc-b59a-f6ae2026dc83 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.631722 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c024b12-0bd4-4a9c-a261-76456ae6ca2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=64118111a1994c3b84aa9d8d8126fddf, user_id=566558040bc34b1bb145df1c76c7a51d, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.635343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c5f49-2df8-4934-846a-acc670fed072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.635889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c024b12-0bd4-4a9c-a261-76456ae6ca2f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.636281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c5f49-2df8-4934-846a-acc670fed072 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.636641 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c5f49-2df8-4934-846a-acc670fed072 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.636991 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c5f49-2df8-4934-846a-acc670fed072 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.638137 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c5f49-2df8-4934-846a-acc670fed072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.639932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c5f49-2df8-4934-846a-acc670fed072 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.654773 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1346/2703] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:37.663808 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c5f49-2df8-4934-846a-acc670fed072 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.677484 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c5f49-2df8-4934-846a-acc670fed072 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.677701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1358/2704] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:37 2022] PUT /identity/v3/projects/64118111a1994c3b84aa9d8d8126fddf/users/566558040bc34b1bb145df1c76c7a51d/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:37.682971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c024b12-0bd4-4a9c-a261-76456ae6ca2f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.787486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c760173-5aab-4337-bdce-e5862fc6e3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.788540 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760173-5aab-4337-bdce-e5862fc6e3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.789748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760173-5aab-4337-bdce-e5862fc6e3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.790027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760173-5aab-4337-bdce-e5862fc6e3f8 admin admin] PATH_INFO: `/v3/projects/e9829b756d4145fdbb9ca28857539187/users/d7b4c6fe7b7a4dd191befa40f77b173c/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.816584 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c760173-5aab-4337-bdce-e5862fc6e3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9829b756d4145fdbb9ca28857539187, user_id=d7b4c6fe7b7a4dd191befa40f77b173c, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.819822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c760173-5aab-4337-bdce-e5862fc6e3f8 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.832712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c024b12-0bd4-4a9c-a261-76456ae6ca2f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.848844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74433b89-ef5d-451e-af1b-8c1b7d60b7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:37.849638 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74433b89-ef5d-451e-af1b-8c1b7d60b7fa admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.850139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74433b89-ef5d-451e-af1b-8c1b7d60b7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.850671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74433b89-ef5d-451e-af1b-8c1b7d60b7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.851852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74433b89-ef5d-451e-af1b-8c1b7d60b7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:37.853562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74433b89-ef5d-451e-af1b-8c1b7d60b7fa admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:37.864766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1347/2705] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:37 2022] PUT /identity/v3/projects/e9829b756d4145fdbb9ca28857539187/users/d7b4c6fe7b7a4dd191befa40f77b173c/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:37.872915 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc46d82-65a3-440a-95a1-5275cd5aeddf None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:37.876212 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc46d82-65a3-440a-95a1-5275cd5aeddf None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:37.876740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc46d82-65a3-440a-95a1-5275cd5aeddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:37.884496 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1359/2706] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:37.885439 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74433b89-ef5d-451e-af1b-8c1b7d60b7fa admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:37.907816 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74433b89-ef5d-451e-af1b-8c1b7d60b7fa admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:37.913205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-acc46d82-65a3-440a-95a1-5275cd5aeddf None None] MFA Rules not processed for user `d7b4c6fe7b7a4dd191befa40f77b173c`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:37.914190 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-acc46d82-65a3-440a-95a1-5275cd5aeddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:38.005513 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27cdde-9d88-40cd-a652-df67906ba556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:38.007347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27cdde-9d88-40cd-a652-df67906ba556 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:38.007803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27cdde-9d88-40cd-a652-df67906ba556 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:38.008181 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27cdde-9d88-40cd-a652-df67906ba556 admin admin] PATH_INFO: `/v3/projects/64118111a1994c3b84aa9d8d8126fddf/users/566558040bc34b1bb145df1c76c7a51d/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:38.011664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27cdde-9d88-40cd-a652-df67906ba556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64118111a1994c3b84aa9d8d8126fddf, user_id=566558040bc34b1bb145df1c76c7a51d, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:38.016850 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1348/2707] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:38.017063 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27cdde-9d88-40cd-a652-df67906ba556 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:38.027848 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc46d82-65a3-440a-95a1-5275cd5aeddf None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:38.032022 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-acc46d82-65a3-440a-95a1-5275cd5aeddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:38.054139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1360/2708] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:37 2022] PUT /identity/v3/projects/64118111a1994c3b84aa9d8d8126fddf/users/566558040bc34b1bb145df1c76c7a51d/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:38.055840 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d34da-717b-40a6-8c45-dafb15f107d0 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:38.056375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d34da-717b-40a6-8c45-dafb15f107d0 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:38.056774 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d34da-717b-40a6-8c45-dafb15f107d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:38.094012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-df7d34da-717b-40a6-8c45-dafb15f107d0 None None] MFA Rules not processed for user `566558040bc34b1bb145df1c76c7a51d`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:38.094952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-df7d34da-717b-40a6-8c45-dafb15f107d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:38.150856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc46d82-65a3-440a-95a1-5275cd5aeddf None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:38.258614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1361/2709] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:38.268229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2de3b8-7dca-4080-a23f-66c3c4aca6da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:38.268995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2de3b8-7dca-4080-a23f-66c3c4aca6da service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:38.269483 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2de3b8-7dca-4080-a23f-66c3c4aca6da service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:38.270050 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2de3b8-7dca-4080-a23f-66c3c4aca6da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:38.276716 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d34da-717b-40a6-8c45-dafb15f107d0 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:38.278809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-df7d34da-717b-40a6-8c45-dafb15f107d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:38.312484 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d34da-717b-40a6-8c45-dafb15f107d0 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:38.319465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e7d5-fb52-4eb7-9222-caecdae8bacd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:38.320134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e7d5-fb52-4eb7-9222-caecdae8bacd service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:38.320418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e7d5-fb52-4eb7-9222-caecdae8bacd service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:38.320694 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e7d5-fb52-4eb7-9222-caecdae8bacd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:38.342211 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70e7d5-fb52-4eb7-9222-caecdae8bacd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:38.346124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70e7d5-fb52-4eb7-9222-caecdae8bacd service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:38.364497 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1362/2710] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:38.389796 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2de3b8-7dca-4080-a23f-66c3c4aca6da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:38.392551 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2de3b8-7dca-4080-a23f-66c3c4aca6da service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:38.410998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1349/2711] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:04:38.481665 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e7d5-fb52-4eb7-9222-caecdae8bacd service cinder] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:38.500371 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2de3b8-7dca-4080-a23f-66c3c4aca6da service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:38.540067 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2de3b8-7dca-4080-a23f-66c3c4aca6da service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:38.579358 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999f496-b6fa-4378-98ba-d619cac3fef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:38.586259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999f496-b6fa-4378-98ba-d619cac3fef2 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:38.589018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999f496-b6fa-4378-98ba-d619cac3fef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:38.590389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999f496-b6fa-4378-98ba-d619cac3fef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:38.774724 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e7d5-fb52-4eb7-9222-caecdae8bacd service cinder] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:38.781713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:38.784293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:38.785883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:38.786555 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] PATH_INFO: `/v3/users/d7b4c6fe7b7a4dd191befa40f77b173c` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:38.804628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7b4c6fe7b7a4dd191befa40f77b173c)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:38.807016 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999f496-b6fa-4378-98ba-d619cac3fef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:38.808591 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999f496-b6fa-4378-98ba-d619cac3fef2 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:38.813030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:38.831787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1350/2712] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:38 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:38.950242 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7b4c6fe7b7a4dd191befa40f77b173c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:39.016499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7b4c6fe7b7a4dd191befa40f77b173c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:39.048007 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7b4c6fe7b7a4dd191befa40f77b173c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:39.049986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7b4c6fe7b7a4dd191befa40f77b173c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:39.081805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7b4c6fe7b7a4dd191befa40f77b173c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:39.101154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1363/2713] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:38 2022] DELETE /identity/v3/users/d7b4c6fe7b7a4dd191befa40f77b173c => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:39.110244 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6583b-f793-4e27-9be7-939eb2755068 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:39.110770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6583b-f793-4e27-9be7-939eb2755068 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:39.111414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6583b-f793-4e27-9be7-939eb2755068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:39.196421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-22b6583b-f793-4e27-9be7-939eb2755068 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:39.201517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-22b6583b-f793-4e27-9be7-939eb2755068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:39.318779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1351/2714] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:39.331710 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:39.333700 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:39.509269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498b8e9-e238-490e-84f4-e1e442709ee8 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:39.514221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e7b2a-7196-4979-be36-89adf782fe5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:39.515055 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e7b2a-7196-4979-be36-89adf782fe5a service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:39.515465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e7b2a-7196-4979-be36-89adf782fe5a service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:39.515812 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e7b2a-7196-4979-be36-89adf782fe5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:39.536844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e7b2a-7196-4979-be36-89adf782fe5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:39.539279 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e7b2a-7196-4979-be36-89adf782fe5a service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:39.557557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1364/2715] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:40.140635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6583b-f793-4e27-9be7-939eb2755068 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:40.142267 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-22b6583b-f793-4e27-9be7-939eb2755068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:40.162503 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6583b-f793-4e27-9be7-939eb2755068 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:40.174544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:40.179794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:40.182066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:40.190809 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] PATH_INFO: `/v3/projects/e9829b756d4145fdbb9ca28857539187` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:40.201265 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9829b756d4145fdbb9ca28857539187)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:40.204290 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:40.379794 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9829b756d4145fdbb9ca28857539187', 'request_id': 'req-893c29cb-0a46-4859-ac04-30df828dc257'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:40.404586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9829b756d4145fdbb9ca28857539187', 'request_id': 'req-893c29cb-0a46-4859-ac04-30df828dc257'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:40.425102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9829b756d4145fdbb9ca28857539187', 'request_id': 'req-893c29cb-0a46-4859-ac04-30df828dc257'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:40.456822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] The token cache is being invalidate because project e9829b756d4145fdbb9ca28857539187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:40.457126 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:40.460292 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-893c29cb-0a46-4859-ac04-30df828dc257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:40.462881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1352/2716] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:40 2022] DELETE /identity/v3/projects/e9829b756d4145fdbb9ca28857539187 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:43.519396 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e7b2a-7196-4979-be36-89adf782fe5a service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:43.523159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c92e7b2a-7196-4979-be36-89adf782fe5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:43.555971 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e7b2a-7196-4979-be36-89adf782fe5a service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:43.667986 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:43.668612 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:43.668801 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:43.669263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] PATH_INFO: `/v3/users/08e5b316aeb5484890ac78972e19221f` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:43.670682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e5b316aeb5484890ac78972e19221f)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:43.673053 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:43.744161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e5b316aeb5484890ac78972e19221f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:43.764451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e5b316aeb5484890ac78972e19221f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:43.776354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e5b316aeb5484890ac78972e19221f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:43.777210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e5b316aeb5484890ac78972e19221f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:43.797378 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e5b316aeb5484890ac78972e19221f'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:43.805640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1365/2717] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:43 2022] DELETE /identity/v3/users/08e5b316aeb5484890ac78972e19221f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:43.811317 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf62f5-4ccb-40e0-87a4-1740bd26d0de None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:43.812733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf62f5-4ccb-40e0-87a4-1740bd26d0de None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:43.812733 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf62f5-4ccb-40e0-87a4-1740bd26d0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:43.834515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-0caf62f5-4ccb-40e0-87a4-1740bd26d0de None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:43.835910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0caf62f5-4ccb-40e0-87a4-1740bd26d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:43.932671 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1353/2718] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:43.943850 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:43.945418 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:44.062763 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1cd7c8-6e2d-47a1-a9b6-bce442c7d9b3 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:44.070121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfcec3-1665-49d4-9f77-b2f390bd967b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:44.071388 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfcec3-1665-49d4-9f77-b2f390bd967b service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:44.071958 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfcec3-1665-49d4-9f77-b2f390bd967b service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:44.072525 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfcec3-1665-49d4-9f77-b2f390bd967b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:44.092772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfcec3-1665-49d4-9f77-b2f390bd967b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:44.095669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfcec3-1665-49d4-9f77-b2f390bd967b service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:44.114805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1366/2719] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:44.602025 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf62f5-4ccb-40e0-87a4-1740bd26d0de None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:44.604387 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0caf62f5-4ccb-40e0-87a4-1740bd26d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:44.646748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf62f5-4ccb-40e0-87a4-1740bd26d0de None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:44.654842 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:44.655835 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:44.656107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:44.656464 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] PATH_INFO: `/v3/projects/06882817e9574bd48f428c200dceb794` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:44.657957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06882817e9574bd48f428c200dceb794)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:44.660615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:44.725336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06882817e9574bd48f428c200dceb794', 'request_id': 'req-97c0b8e2-339a-4b13-ae26-afe05eb004f8'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:44.737747 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06882817e9574bd48f428c200dceb794', 'request_id': 'req-97c0b8e2-339a-4b13-ae26-afe05eb004f8'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:44.745190 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06882817e9574bd48f428c200dceb794', 'request_id': 'req-97c0b8e2-339a-4b13-ae26-afe05eb004f8'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:44.763002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] The token cache is being invalidate because project 06882817e9574bd48f428c200dceb794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:44.763384 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:44.764215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:44.766184 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1354/2720] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:44 2022] DELETE /identity/v3/projects/06882817e9574bd48f428c200dceb794 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:44.771900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfcec3-1665-49d4-9f77-b2f390bd967b service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:44.791453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfcec3-1665-49d4-9f77-b2f390bd967b service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:44.879247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:44.880182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:44.880527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:44.880876 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] PATH_INFO: `/v3/users/566558040bc34b1bb145df1c76c7a51d` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:44.882190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=566558040bc34b1bb145df1c76c7a51d)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:44.885016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:44.968575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566558040bc34b1bb145df1c76c7a51d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:44.994805 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566558040bc34b1bb145df1c76c7a51d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:45.002870 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566558040bc34b1bb145df1c76c7a51d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:45.003965 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566558040bc34b1bb145df1c76c7a51d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:45.016037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-c651643c-b1f0-46c1-9f83-8ba85c571d68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566558040bc34b1bb145df1c76c7a51d'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:45.025811 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1367/2721] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:44 2022] DELETE /identity/v3/users/566558040bc34b1bb145df1c76c7a51d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:45.517341 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:45.519614 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:45.559373 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0b8e2-339a-4b13-ae26-afe05eb004f8 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:45.714173 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:45.715526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:45.715526 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:45.715875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] PATH_INFO: `/v3/projects/64118111a1994c3b84aa9d8d8126fddf` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:45.717027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64118111a1994c3b84aa9d8d8126fddf)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:45.719644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:45.803860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64118111a1994c3b84aa9d8d8126fddf', 'request_id': 'req-febc74b0-523a-4c18-9953-5c3416458108'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:45.815916 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64118111a1994c3b84aa9d8d8126fddf', 'request_id': 'req-febc74b0-523a-4c18-9953-5c3416458108'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:45.825677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64118111a1994c3b84aa9d8d8126fddf', 'request_id': 'req-febc74b0-523a-4c18-9953-5c3416458108'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:45.849150 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] The token cache is being invalidate because project 64118111a1994c3b84aa9d8d8126fddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:45.849532 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:45.850666 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-febc74b0-523a-4c18-9953-5c3416458108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:45.859464 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1355/2722] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:45 2022] DELETE /identity/v3/projects/64118111a1994c3b84aa9d8d8126fddf => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:45.861144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73300efb-e05a-4ab4-b716-681962d91e50 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:45.861671 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73300efb-e05a-4ab4-b716-681962d91e50 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:45.861800 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73300efb-e05a-4ab4-b716-681962d91e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:45.891885 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-73300efb-e05a-4ab4-b716-681962d91e50 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:45.894183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-73300efb-e05a-4ab4-b716-681962d91e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:46.024261 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1368/2723] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:46.026114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769aa08e-01c7-4acf-a049-a7a61f7b8e6d None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:46.026404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769aa08e-01c7-4acf-a049-a7a61f7b8e6d None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:46.026686 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769aa08e-01c7-4acf-a049-a7a61f7b8e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:46.057058 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-769aa08e-01c7-4acf-a049-a7a61f7b8e6d None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:46.058610 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-769aa08e-01c7-4acf-a049-a7a61f7b8e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:46.078435 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1356/2724] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:46.080417 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73300efb-e05a-4ab4-b716-681962d91e50 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:46.096974 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73300efb-e05a-4ab4-b716-681962d91e50 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:46.103010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e6252-1ef1-422f-a1bf-fff8b4524014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:46.104453 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e6252-1ef1-422f-a1bf-fff8b4524014 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:46.105068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e6252-1ef1-422f-a1bf-fff8b4524014 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:46.107967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e6252-1ef1-422f-a1bf-fff8b4524014 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:46.111783 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e6252-1ef1-422f-a1bf-fff8b4524014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:46.112100 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e6252-1ef1-422f-a1bf-fff8b4524014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:46.113776 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e6252-1ef1-422f-a1bf-fff8b4524014 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:46.129486 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1369/2725] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:04:46.134332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769aa08e-01c7-4acf-a049-a7a61f7b8e6d None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:46.149668 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769aa08e-01c7-4acf-a049-a7a61f7b8e6d None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:46.156701 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf788510-3808-4386-92d6-2e0b88f88194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:46.157458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788510-3808-4386-92d6-2e0b88f88194 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:46.158027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788510-3808-4386-92d6-2e0b88f88194 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:46.158027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788510-3808-4386-92d6-2e0b88f88194 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:46.158806 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf788510-3808-4386-92d6-2e0b88f88194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:46.160769 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf788510-3808-4386-92d6-2e0b88f88194 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:46.198415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1357/2726] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:46 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:46.200038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e6252-1ef1-422f-a1bf-fff8b4524014 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:46.213472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e6252-1ef1-422f-a1bf-fff8b4524014 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:46.290657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599aa8b-5b4b-47b7-a072-56a5662f1aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:46.292087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599aa8b-5b4b-47b7-a072-56a5662f1aae admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:46.292476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599aa8b-5b4b-47b7-a072-56a5662f1aae admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:46.295839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599aa8b-5b4b-47b7-a072-56a5662f1aae admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:46.297528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599aa8b-5b4b-47b7-a072-56a5662f1aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:46.300311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599aa8b-5b4b-47b7-a072-56a5662f1aae admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:46.337470 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1370/2727] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:46 2022] POST /identity/v3/users => generated 284 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:46.339167 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf788510-3808-4386-92d6-2e0b88f88194 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:46.350764 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf788510-3808-4386-92d6-2e0b88f88194 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:46.356832 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429674a-e92b-4cdb-b4db-4030175ec584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:46.357740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429674a-e92b-4cdb-b4db-4030175ec584 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:46.358026 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429674a-e92b-4cdb-b4db-4030175ec584 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:46.358301 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429674a-e92b-4cdb-b4db-4030175ec584 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:46.359693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429674a-e92b-4cdb-b4db-4030175ec584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:46.361378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429674a-e92b-4cdb-b4db-4030175ec584 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:46.384507 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1358/2728] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:04:46.387756 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599aa8b-5b4b-47b7-a072-56a5662f1aae admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:46.398927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599aa8b-5b4b-47b7-a072-56a5662f1aae admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:46.414057 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72287e75-ef12-4543-9ed7-6f22fd2b9ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:46.414945 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72287e75-ef12-4543-9ed7-6f22fd2b9ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:46.415339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72287e75-ef12-4543-9ed7-6f22fd2b9ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:46.415709 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72287e75-ef12-4543-9ed7-6f22fd2b9ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:46.418969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72287e75-ef12-4543-9ed7-6f22fd2b9ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:46.420771 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72287e75-ef12-4543-9ed7-6f22fd2b9ec2 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:46.434531 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1371/2729] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:46.443351 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429674a-e92b-4cdb-b4db-4030175ec584 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:46.455523 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429674a-e92b-4cdb-b4db-4030175ec584 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:46.465107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d7388-3574-4129-b734-1d7735372ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:46.467505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d7388-3574-4129-b734-1d7735372ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:46.467986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d7388-3574-4129-b734-1d7735372ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:46.469142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d7388-3574-4129-b734-1d7735372ba1 admin admin] PATH_INFO: `/v3/projects/a07e699eb8454f3d93711ccc62cff4f1/users/0c30b8c772a24984aa46e5e61247af34/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:46.487079 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d7388-3574-4129-b734-1d7735372ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07e699eb8454f3d93711ccc62cff4f1, user_id=0c30b8c772a24984aa46e5e61247af34, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:46.490009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d7388-3574-4129-b734-1d7735372ba1 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:46.506895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1359/2730] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:46 2022] PUT /identity/v3/projects/a07e699eb8454f3d93711ccc62cff4f1/users/0c30b8c772a24984aa46e5e61247af34/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:46.519433 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08596d-b2b3-4ede-b503-edfee17a67ea None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:46.520701 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08596d-b2b3-4ede-b503-edfee17a67ea None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:46.521174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08596d-b2b3-4ede-b503-edfee17a67ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:46.562310 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-6b08596d-b2b3-4ede-b503-edfee17a67ea None None] MFA Rules not processed for user `0c30b8c772a24984aa46e5e61247af34`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:46.563516 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6b08596d-b2b3-4ede-b503-edfee17a67ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:46.662441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1372/2731] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:46.664091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d7388-3574-4129-b734-1d7735372ba1 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:46.765702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d7388-3574-4129-b734-1d7735372ba1 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:46.771544 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1df60e-9b1f-460e-acb1-2e18c393d041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:46.772353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1df60e-9b1f-460e-acb1-2e18c393d041 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:46.772697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1df60e-9b1f-460e-acb1-2e18c393d041 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:46.773037 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1df60e-9b1f-460e-acb1-2e18c393d041 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:46.773802 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1df60e-9b1f-460e-acb1-2e18c393d041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:46.776038 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1df60e-9b1f-460e-acb1-2e18c393d041 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:46.825107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1360/2732] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:46 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:46.826383 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08596d-b2b3-4ede-b503-edfee17a67ea None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:46.839507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08596d-b2b3-4ede-b503-edfee17a67ea None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:46.915849 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c893eb9-7ffd-4653-9a6c-23a3eb211ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:46.916641 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893eb9-7ffd-4653-9a6c-23a3eb211ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:46.917119 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893eb9-7ffd-4653-9a6c-23a3eb211ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:46.917780 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c893eb9-7ffd-4653-9a6c-23a3eb211ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:46.919010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c893eb9-7ffd-4653-9a6c-23a3eb211ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:46.923894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c893eb9-7ffd-4653-9a6c-23a3eb211ca5 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:46.956918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1373/2733] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:46 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:46.963087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1df60e-9b1f-460e-acb1-2e18c393d041 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:46.989501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1df60e-9b1f-460e-acb1-2e18c393d041 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:46.998496 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d116a0-ed9c-489c-b51b-10832ffe9ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:47.000280 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d116a0-ed9c-489c-b51b-10832ffe9ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:47.000748 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d116a0-ed9c-489c-b51b-10832ffe9ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:47.001259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d116a0-ed9c-489c-b51b-10832ffe9ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:47.002568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d116a0-ed9c-489c-b51b-10832ffe9ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:47.005297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d116a0-ed9c-489c-b51b-10832ffe9ca0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:47.025185 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1361/2734] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:47.030266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c893eb9-7ffd-4653-9a6c-23a3eb211ca5 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:47.031995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d116a0-ed9c-489c-b51b-10832ffe9ca0 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:47.034291 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a8d116a0-ed9c-489c-b51b-10832ffe9ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:47.045721 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c893eb9-7ffd-4653-9a6c-23a3eb211ca5 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:47.050640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d116a0-ed9c-489c-b51b-10832ffe9ca0 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:47.052033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f441a5f-fd22-43db-b155-53200fa4612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:47.052937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f441a5f-fd22-43db-b155-53200fa4612c admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:47.053291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f441a5f-fd22-43db-b155-53200fa4612c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:47.053502 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f441a5f-fd22-43db-b155-53200fa4612c admin admin] PATH_INFO: `/v3/projects/2b4a69873f1d4ec0a7dfdb3a93701914/users/e877723a48b346f69474ccebe9306eb9/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:47.059304 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:47.059895 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:47.060568 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:47.061077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] PATH_INFO: `/v3/users/b45217b58eab41c1872275a52d0dac2d` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:47.062629 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b45217b58eab41c1872275a52d0dac2d)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:47.064426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:47.083253 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f441a5f-fd22-43db-b155-53200fa4612c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4a69873f1d4ec0a7dfdb3a93701914, user_id=e877723a48b346f69474ccebe9306eb9, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:47.086623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f441a5f-fd22-43db-b155-53200fa4612c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:47.107471 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1374/2735] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:47 2022] PUT /identity/v3/projects/2b4a69873f1d4ec0a7dfdb3a93701914/users/e877723a48b346f69474ccebe9306eb9/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:47.109398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f441a5f-fd22-43db-b155-53200fa4612c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:47.132039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b45217b58eab41c1872275a52d0dac2d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:47.133348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b45217b58eab41c1872275a52d0dac2d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:47.148989 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b45217b58eab41c1872275a52d0dac2d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:47.149760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b45217b58eab41c1872275a52d0dac2d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:47.160742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-09b355a7-7fbb-4f81-bc33-832784decf0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b45217b58eab41c1872275a52d0dac2d'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:47.186386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1362/2736] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:47 2022] DELETE /identity/v3/users/b45217b58eab41c1872275a52d0dac2d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:47.196698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd4f1d-d73c-406e-9519-67def72942fc None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:47.197658 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd4f1d-d73c-406e-9519-67def72942fc None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:47.198632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd4f1d-d73c-406e-9519-67def72942fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:47.230343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c1bd4f1d-d73c-406e-9519-67def72942fc None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:47.232874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c1bd4f1d-d73c-406e-9519-67def72942fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:47.237182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f441a5f-fd22-43db-b155-53200fa4612c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:47.348117 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1363/2737] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:47.363077 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd4f1d-d73c-406e-9519-67def72942fc None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:47.365435 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c1bd4f1d-d73c-406e-9519-67def72942fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:47.378386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d251205-fedf-45c8-9c2e-f35eec2649f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:47.379308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d251205-fedf-45c8-9c2e-f35eec2649f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:47.379918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d251205-fedf-45c8-9c2e-f35eec2649f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:47.380341 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d251205-fedf-45c8-9c2e-f35eec2649f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:47.381503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d251205-fedf-45c8-9c2e-f35eec2649f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:47.385514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d251205-fedf-45c8-9c2e-f35eec2649f3 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:47.415491 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1375/2738] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:47.419793 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d251205-fedf-45c8-9c2e-f35eec2649f3 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:47.430924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9d251205-fedf-45c8-9c2e-f35eec2649f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:47.448377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d251205-fedf-45c8-9c2e-f35eec2649f3 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:47.458244 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd4f1d-d73c-406e-9519-67def72942fc None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:47.461652 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f6db2-d12c-49c4-b42c-914fd2a04551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:47.462569 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e1d30-a821-4d37-ac1e-292de06dc038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:47.462747 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6db2-d12c-49c4-b42c-914fd2a04551 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:47.462969 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6db2-d12c-49c4-b42c-914fd2a04551 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:47.463180 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6db2-d12c-49c4-b42c-914fd2a04551 admin admin] PATH_INFO: `/v3/projects/2b4a69873f1d4ec0a7dfdb3a93701914/users/e877723a48b346f69474ccebe9306eb9/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:47.463406 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e1d30-a821-4d37-ac1e-292de06dc038 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:47.463695 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e1d30-a821-4d37-ac1e-292de06dc038 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:47.464003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e1d30-a821-4d37-ac1e-292de06dc038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:47.467849 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f6db2-d12c-49c4-b42c-914fd2a04551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4a69873f1d4ec0a7dfdb3a93701914, user_id=e877723a48b346f69474ccebe9306eb9, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:47.471163 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f6db2-d12c-49c4-b42c-914fd2a04551 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:47.479338 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e1d30-a821-4d37-ac1e-292de06dc038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:47.480807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e1d30-a821-4d37-ac1e-292de06dc038 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:47.486288 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1376/2739] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:47 2022] PUT /identity/v3/projects/2b4a69873f1d4ec0a7dfdb3a93701914/users/e877723a48b346f69474ccebe9306eb9/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:47.499068 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177980d-04a1-4e6b-8a1f-29bf85758db8 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:47.499906 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177980d-04a1-4e6b-8a1f-29bf85758db8 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:47.500589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177980d-04a1-4e6b-8a1f-29bf85758db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:47.506495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1364/2740] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:04:47.549151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-3177980d-04a1-4e6b-8a1f-29bf85758db8 None None] MFA Rules not processed for user `e877723a48b346f69474ccebe9306eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:47.551597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-3177980d-04a1-4e6b-8a1f-29bf85758db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:47.701244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1377/2741] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:47.711840 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e1d30-a821-4d37-ac1e-292de06dc038 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:47.714200 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2d7e1d30-a821-4d37-ac1e-292de06dc038 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:47.791385 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e1d30-a821-4d37-ac1e-292de06dc038 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:47.851366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686ae9-e9d1-4e58-9675-87fdf10d1782 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:47.851714 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686ae9-e9d1-4e58-9675-87fdf10d1782 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:47.851945 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686ae9-e9d1-4e58-9675-87fdf10d1782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:47.875784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696b0ee-5f1d-4426-8220-33c69f79d931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:47.876482 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696b0ee-5f1d-4426-8220-33c69f79d931 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:47.876758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f4686ae9-e9d1-4e58-9675-87fdf10d1782 None None] MFA Rules not processed for user `3bbfb51ebb5f4553a9ef7b70becf1a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:47.876869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696b0ee-5f1d-4426-8220-33c69f79d931 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:47.877091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696b0ee-5f1d-4426-8220-33c69f79d931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:47.878017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f4686ae9-e9d1-4e58-9675-87fdf10d1782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:47.899475 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696b0ee-5f1d-4426-8220-33c69f79d931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:47.902041 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696b0ee-5f1d-4426-8220-33c69f79d931 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:47.914485 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1365/2742] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:48.025831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1378/2743] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:48.056723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696b0ee-5f1d-4426-8220-33c69f79d931 service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:48.069822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696b0ee-5f1d-4426-8220-33c69f79d931 service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:48.076481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bba67-9c00-41ff-89c2-aae7c6d3d07c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:48.078269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bba67-9c00-41ff-89c2-aae7c6d3d07c service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:48.078926 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bba67-9c00-41ff-89c2-aae7c6d3d07c service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:48.079511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bba67-9c00-41ff-89c2-aae7c6d3d07c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:48.083075 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5e7bba67-9c00-41ff-89c2-aae7c6d3d07c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:48.127727 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4686ae9-e9d1-4e58-9675-87fdf10d1782 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:48.129203 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f4686ae9-e9d1-4e58-9675-87fdf10d1782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:48.221023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bba67-9c00-41ff-89c2-aae7c6d3d07c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:48.222480 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bba67-9c00-41ff-89c2-aae7c6d3d07c service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:48.235936 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1366/2744] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:48.288449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4686ae9-e9d1-4e58-9675-87fdf10d1782 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:48.306167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:48.307712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:48.308411 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:48.309350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] PATH_INFO: `/v3/projects/8a7084147cb34bdfa8c72a5de3b84ea9` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:48.311938 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7084147cb34bdfa8c72a5de3b84ea9)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:48.317819 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:48.413184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7084147cb34bdfa8c72a5de3b84ea9', 'request_id': 'req-fcf7534f-4f47-4834-bcd4-cdea9838acf3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:48.428581 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7084147cb34bdfa8c72a5de3b84ea9', 'request_id': 'req-fcf7534f-4f47-4834-bcd4-cdea9838acf3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:48.443176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7084147cb34bdfa8c72a5de3b84ea9', 'request_id': 'req-fcf7534f-4f47-4834-bcd4-cdea9838acf3'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:48.466016 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] The token cache is being invalidate because project 8a7084147cb34bdfa8c72a5de3b84ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:48.466355 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:48.467274 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-fcf7534f-4f47-4834-bcd4-cdea9838acf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:48.470210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1379/2745] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:48 2022] DELETE /identity/v3/projects/8a7084147cb34bdfa8c72a5de3b84ea9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:48.476976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b8122-c93c-430f-a6ad-aa44fc865a22 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:48.477641 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b8122-c93c-430f-a6ad-aa44fc865a22 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:48.478762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b8122-c93c-430f-a6ad-aa44fc865a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:48.511837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-809b8122-c93c-430f-a6ad-aa44fc865a22 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:48.513504 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-809b8122-c93c-430f-a6ad-aa44fc865a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:48.613293 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1367/2746] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:48.625432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7da3-0a8d-43b3-a653-774c9dcf19fe None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:48.628054 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7da3-0a8d-43b3-a653-774c9dcf19fe None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:48.629138 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7da3-0a8d-43b3-a653-774c9dcf19fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:48.670451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-32db7da3-0a8d-43b3-a653-774c9dcf19fe None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:48.673323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-32db7da3-0a8d-43b3-a653-774c9dcf19fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:48.691342 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1380/2747] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:48.693461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b8122-c93c-430f-a6ad-aa44fc865a22 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:48.729366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b8122-c93c-430f-a6ad-aa44fc865a22 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:48.738632 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461456e2-58fa-4199-9085-a1675b478911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:48.739862 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461456e2-58fa-4199-9085-a1675b478911 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:48.740560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461456e2-58fa-4199-9085-a1675b478911 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:48.740826 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461456e2-58fa-4199-9085-a1675b478911 admin admin] PATH_INFO: `/v3/domains` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:48.744192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461456e2-58fa-4199-9085-a1675b478911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:48.744192 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461456e2-58fa-4199-9085-a1675b478911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:48.748560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461456e2-58fa-4199-9085-a1675b478911 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:48.765149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1368/2748] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 14:04:48.769715 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db7da3-0a8d-43b3-a653-774c9dcf19fe None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:48.787595 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db7da3-0a8d-43b3-a653-774c9dcf19fe None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:48.797210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0478a51-2af0-453f-87f3-0e5967106f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:48.797898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0478a51-2af0-453f-87f3-0e5967106f6e admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:48.798169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0478a51-2af0-453f-87f3-0e5967106f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:48.798543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0478a51-2af0-453f-87f3-0e5967106f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:48.799921 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0478a51-2af0-453f-87f3-0e5967106f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:48.803050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0478a51-2af0-453f-87f3-0e5967106f6e admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:48.834674 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1381/2749] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:48 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:04:48.839084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461456e2-58fa-4199-9085-a1675b478911 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:48.850878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461456e2-58fa-4199-9085-a1675b478911 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:48.911647 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e067d6-f556-4d25-91f5-9f27300d4c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:48.912329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e067d6-f556-4d25-91f5-9f27300d4c9b admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:48.912605 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e067d6-f556-4d25-91f5-9f27300d4c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:48.912883 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e067d6-f556-4d25-91f5-9f27300d4c9b admin admin] PATH_INFO: `/v3/users` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:48.913564 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e067d6-f556-4d25-91f5-9f27300d4c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:48.915560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e067d6-f556-4d25-91f5-9f27300d4c9b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:48.940810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1369/2750] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:48 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:48.945367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0478a51-2af0-453f-87f3-0e5967106f6e admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:48.957710 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0478a51-2af0-453f-87f3-0e5967106f6e admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:48.963365 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a06eaf-a0eb-441c-a6e6-caba99a91e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:48.964044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a06eaf-a0eb-441c-a6e6-caba99a91e05 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:48.964323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a06eaf-a0eb-441c-a6e6-caba99a91e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:48.964587 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a06eaf-a0eb-441c-a6e6-caba99a91e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:48.965585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a06eaf-a0eb-441c-a6e6-caba99a91e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:48.967514 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a06eaf-a0eb-441c-a6e6-caba99a91e05 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:48.985830 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1382/2751] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:48.988891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e067d6-f556-4d25-91f5-9f27300d4c9b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:49.006669 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e067d6-f556-4d25-91f5-9f27300d4c9b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:49.015061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bd412-8ee1-44cf-9fb8-626e206dc66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:49.016121 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bd412-8ee1-44cf-9fb8-626e206dc66a admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:49.016583 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bd412-8ee1-44cf-9fb8-626e206dc66a admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:49.017019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bd412-8ee1-44cf-9fb8-626e206dc66a admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:49.018198 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bd412-8ee1-44cf-9fb8-626e206dc66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:49.020515 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bd412-8ee1-44cf-9fb8-626e206dc66a admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:49.032343 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1370/2752] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:49.036681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a06eaf-a0eb-441c-a6e6-caba99a91e05 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:49.048630 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a06eaf-a0eb-441c-a6e6-caba99a91e05 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:49.053713 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea87d82-138e-4d99-837f-5586c8551cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:49.054817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea87d82-138e-4d99-837f-5586c8551cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:49.055177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea87d82-138e-4d99-837f-5586c8551cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:49.055528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea87d82-138e-4d99-837f-5586c8551cb1 admin admin] PATH_INFO: `/v3/projects/9ec09ba22d804f168c25b95f1636d325/users/ec486e4fd685475bb903baac25cdf746/roles/90868071f00340ae853f1188777a2244` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:49.078247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea87d82-138e-4d99-837f-5586c8551cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec09ba22d804f168c25b95f1636d325, user_id=ec486e4fd685475bb903baac25cdf746, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:49.082242 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea87d82-138e-4d99-837f-5586c8551cb1 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:49.100681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1383/2753] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:49 2022] PUT /identity/v3/projects/9ec09ba22d804f168c25b95f1636d325/users/ec486e4fd685475bb903baac25cdf746/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:49.117973 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80881ca8-ee24-43e2-9bbf-7015b2282e3e None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:49.118270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80881ca8-ee24-43e2-9bbf-7015b2282e3e None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:49.118555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80881ca8-ee24-43e2-9bbf-7015b2282e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:49.148923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea87d82-138e-4d99-837f-5586c8551cb1 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:49.151300 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-5ea87d82-138e-4d99-837f-5586c8551cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:49.152362 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-80881ca8-ee24-43e2-9bbf-7015b2282e3e None None] MFA Rules not processed for user `ec486e4fd685475bb903baac25cdf746`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:49.153815 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-80881ca8-ee24-43e2-9bbf-7015b2282e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:49.238188 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea87d82-138e-4d99-837f-5586c8551cb1 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:49.238867 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1371/2754] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:04:49.248006 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdec95-2675-4c1b-b431-44c07dd31ba9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ae9a6f1ac304e21b8d3115f2a0baf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:49.248620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80881ca8-ee24-43e2-9bbf-7015b2282e3e None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:49.249112 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdec95-2675-4c1b-b431-44c07dd31ba9 service placement] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:49.249642 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdec95-2675-4c1b-b431-44c07dd31ba9 service placement] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:49.249957 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdec95-2675-4c1b-b431-44c07dd31ba9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:49.250130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-80881ca8-ee24-43e2-9bbf-7015b2282e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:49.252027 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-efbdec95-2675-4c1b-b431-44c07dd31ba9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:49.267145 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdec95-2675-4c1b-b431-44c07dd31ba9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:49.269357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdec95-2675-4c1b-b431-44c07dd31ba9 service placement] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:49.287759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1384/2755] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:04:49.319374 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80881ca8-ee24-43e2-9bbf-7015b2282e3e None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:49.331637 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5efcd-d857-45fa-9ad7-f93c2a456e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:49.332702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5efcd-d857-45fa-9ad7-f93c2a456e5f service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:49.333164 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5efcd-d857-45fa-9ad7-f93c2a456e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:49.333646 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5efcd-d857-45fa-9ad7-f93c2a456e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:49.347877 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5efcd-d857-45fa-9ad7-f93c2a456e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:49.349468 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5efcd-d857-45fa-9ad7-f93c2a456e5f service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:49.361329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1372/2756] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:04:50.734107 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a67d-a648-4e22-986d-55587df11fa3 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:50.734672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a67d-a648-4e22-986d-55587df11fa3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:50.734672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a67d-a648-4e22-986d-55587df11fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:50.752436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-79e7a67d-a648-4e22-986d-55587df11fa3 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:50.753661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-79e7a67d-a648-4e22-986d-55587df11fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:50.827033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1385/2757] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 14:04:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:50.841745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5efcd-d857-45fa-9ad7-f93c2a456e5f service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:50.843513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-def5efcd-d857-45fa-9ad7-f93c2a456e5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:50.938989 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5efcd-d857-45fa-9ad7-f93c2a456e5f service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:50.944460 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402ed50-9119-4bbe-8b0e-4217ce933b8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:50.945120 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ed50-9119-4bbe-8b0e-4217ce933b8c service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:50.945398 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ed50-9119-4bbe-8b0e-4217ce933b8c service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:50.945676 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ed50-9119-4bbe-8b0e-4217ce933b8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:50.956683 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402ed50-9119-4bbe-8b0e-4217ce933b8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:50.959002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402ed50-9119-4bbe-8b0e-4217ce933b8c service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:50.969309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1373/2758] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:04:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 14:04:53.986234 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7a67d-a648-4e22-986d-55587df11fa3 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:53.994678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-79e7a67d-a648-4e22-986d-55587df11fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:54.112923 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7a67d-a648-4e22-986d-55587df11fa3 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:54.117775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:54.118647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:54.118967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:54.119249 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] PATH_INFO: `/v3/users/0c30b8c772a24984aa46e5e61247af34` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:54.121037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c30b8c772a24984aa46e5e61247af34)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:54.123265 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:54.173760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c30b8c772a24984aa46e5e61247af34'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:54.191085 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c30b8c772a24984aa46e5e61247af34'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:54.198975 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c30b8c772a24984aa46e5e61247af34'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:54.199838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c30b8c772a24984aa46e5e61247af34'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:54.209773 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c30b8c772a24984aa46e5e61247af34'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:54.217634 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1386/2759] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:53 2022] DELETE /identity/v3/users/0c30b8c772a24984aa46e5e61247af34 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:54.222396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae986e-8fcb-4325-a659-e5501cb0440a None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:54.222994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae986e-8fcb-4325-a659-e5501cb0440a None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:54.222994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae986e-8fcb-4325-a659-e5501cb0440a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:54.253594 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-fbae986e-8fcb-4325-a659-e5501cb0440a None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:54.255562 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-fbae986e-8fcb-4325-a659-e5501cb0440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:54.360365 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1374/2760] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:54.367367 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:54.369362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:54.473407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42bba5-fc50-4362-ba90-f4d630d70815 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:54.481146 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a54819-fc47-4ab2-ad9e-29006474b464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:54.481818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a54819-fc47-4ab2-ad9e-29006474b464 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:54.482124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a54819-fc47-4ab2-ad9e-29006474b464 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:54.482379 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a54819-fc47-4ab2-ad9e-29006474b464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:54.502217 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a54819-fc47-4ab2-ad9e-29006474b464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:54.503791 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a54819-fc47-4ab2-ad9e-29006474b464 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:54.547184 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1387/2761] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:04:55.066899 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae986e-8fcb-4325-a659-e5501cb0440a None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:55.068450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-fbae986e-8fcb-4325-a659-e5501cb0440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:55.091186 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae986e-8fcb-4325-a659-e5501cb0440a None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:55.118102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:55.118985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:55.119273 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:55.119711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] PATH_INFO: `/v3/projects/a07e699eb8454f3d93711ccc62cff4f1` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:55.122467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07e699eb8454f3d93711ccc62cff4f1)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:55.125517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:55.263241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07e699eb8454f3d93711ccc62cff4f1', 'request_id': 'req-f486f3b1-a498-4d74-b69c-bfead4d2bd26'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:55.271575 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07e699eb8454f3d93711ccc62cff4f1', 'request_id': 'req-f486f3b1-a498-4d74-b69c-bfead4d2bd26'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:55.279329 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07e699eb8454f3d93711ccc62cff4f1', 'request_id': 'req-f486f3b1-a498-4d74-b69c-bfead4d2bd26'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:55.312739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] The token cache is being invalidate because project a07e699eb8454f3d93711ccc62cff4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:55.312739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:55.312739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:55.312739 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1375/2762] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:55 2022] DELETE /identity/v3/projects/a07e699eb8454f3d93711ccc62cff4f1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:55.314220 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a54819-fc47-4ab2-ad9e-29006474b464 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:55.337402 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a54819-fc47-4ab2-ad9e-29006474b464 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:55.440716 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:55.441633 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:55.442109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:55.442549 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] PATH_INFO: `/v3/users/e877723a48b346f69474ccebe9306eb9` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:55.444282 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e877723a48b346f69474ccebe9306eb9)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:55.446401 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:55.525253 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e877723a48b346f69474ccebe9306eb9'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:55.548782 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e877723a48b346f69474ccebe9306eb9'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:55.568339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e877723a48b346f69474ccebe9306eb9'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:55.569124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e877723a48b346f69474ccebe9306eb9'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:55.583617 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-62aca98b-16ba-4477-b173-fe21e68b7193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e877723a48b346f69474ccebe9306eb9'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:55.606436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1388/2763] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:04:55 2022] DELETE /identity/v3/users/e877723a48b346f69474ccebe9306eb9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:56.038982 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:56.040693 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:56.066048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486f3b1-a498-4d74-b69c-bfead4d2bd26 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:56.168001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:56.169021 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:56.169417 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:56.169782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] PATH_INFO: `/v3/projects/2b4a69873f1d4ec0a7dfdb3a93701914` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:56.172226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b4a69873f1d4ec0a7dfdb3a93701914)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:56.174784 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:56.253288 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b4a69873f1d4ec0a7dfdb3a93701914', 'request_id': 'req-53bcfdd9-e101-4686-a585-5cafe448f037'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:56.259858 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b4a69873f1d4ec0a7dfdb3a93701914', 'request_id': 'req-53bcfdd9-e101-4686-a585-5cafe448f037'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:56.265725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b4a69873f1d4ec0a7dfdb3a93701914', 'request_id': 'req-53bcfdd9-e101-4686-a585-5cafe448f037'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:56.285104 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] The token cache is being invalidate because project 2b4a69873f1d4ec0a7dfdb3a93701914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:04:56.285607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:56.286505 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-53bcfdd9-e101-4686-a585-5cafe448f037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:04:56.288829 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1376/2764] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:04:56 2022] DELETE /identity/v3/projects/2b4a69873f1d4ec0a7dfdb3a93701914 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:56.293708 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9054bec-27a3-488a-bddc-bf39c01b5d36 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:56.294058 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9054bec-27a3-488a-bddc-bf39c01b5d36 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:56.294263 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9054bec-27a3-488a-bddc-bf39c01b5d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:56.319589 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a9054bec-27a3-488a-bddc-bf39c01b5d36 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:56.323832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a9054bec-27a3-488a-bddc-bf39c01b5d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:56.418750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1389/2765] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:56.436178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d95d7c-9263-42ef-ab91-8af0848e702b None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:56.436998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d95d7c-9263-42ef-ab91-8af0848e702b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:56.437439 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d95d7c-9263-42ef-ab91-8af0848e702b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:56.468559 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-39d95d7c-9263-42ef-ab91-8af0848e702b None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:56.469822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-39d95d7c-9263-42ef-ab91-8af0848e702b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:56.489790 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1377/2766] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:56.491386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9054bec-27a3-488a-bddc-bf39c01b5d36 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:56.503284 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9054bec-27a3-488a-bddc-bf39c01b5d36 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:56.508189 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874cb2d-7ef4-4b58-9d7d-79f849248a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:56.509695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874cb2d-7ef4-4b58-9d7d-79f849248a98 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:56.510195 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874cb2d-7ef4-4b58-9d7d-79f849248a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:56.510758 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874cb2d-7ef4-4b58-9d7d-79f849248a98 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:56.511738 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874cb2d-7ef4-4b58-9d7d-79f849248a98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:04:56.512314 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874cb2d-7ef4-4b58-9d7d-79f849248a98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:56.514190 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874cb2d-7ef4-4b58-9d7d-79f849248a98 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:56.531405 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1390/2767] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 14:04:56.536974 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d95d7c-9263-42ef-ab91-8af0848e702b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:56.566818 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d95d7c-9263-42ef-ab91-8af0848e702b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:56.573779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f3344-b792-4764-815b-c009b130cb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:56.574976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f3344-b792-4764-815b-c009b130cb68 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:56.575506 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f3344-b792-4764-815b-c009b130cb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:56.576003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f3344-b792-4764-815b-c009b130cb68 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:56.577319 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f3344-b792-4764-815b-c009b130cb68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:56.580673 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f3344-b792-4764-815b-c009b130cb68 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:56.633499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1378/2768] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:56 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:56.639329 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874cb2d-7ef4-4b58-9d7d-79f849248a98 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:56.657872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874cb2d-7ef4-4b58-9d7d-79f849248a98 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:56.750656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a219cd-3df9-4b22-bfe3-2e3e67be1cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:56.751650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a219cd-3df9-4b22-bfe3-2e3e67be1cad admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:56.752033 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a219cd-3df9-4b22-bfe3-2e3e67be1cad admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:56.752407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a219cd-3df9-4b22-bfe3-2e3e67be1cad admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:56.753370 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a219cd-3df9-4b22-bfe3-2e3e67be1cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:56.755893 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a219cd-3df9-4b22-bfe3-2e3e67be1cad admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:56.789182 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1391/2769] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:56 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:04:56.793511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f3344-b792-4764-815b-c009b130cb68 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:56.806082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f3344-b792-4764-815b-c009b130cb68 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:56.818305 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35425f14-18dc-4828-a074-58a242290c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:56.820180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35425f14-18dc-4828-a074-58a242290c77 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:56.820180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35425f14-18dc-4828-a074-58a242290c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:56.820180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35425f14-18dc-4828-a074-58a242290c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:56.820855 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35425f14-18dc-4828-a074-58a242290c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:56.823113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35425f14-18dc-4828-a074-58a242290c77 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:56.834770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1379/2770] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:56.838804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a219cd-3df9-4b22-bfe3-2e3e67be1cad admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:56.849573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a219cd-3df9-4b22-bfe3-2e3e67be1cad admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:56.871034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c1786-71c7-408d-b8b1-f474b2ead7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:56.871034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c1786-71c7-408d-b8b1-f474b2ead7dc admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:56.871034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c1786-71c7-408d-b8b1-f474b2ead7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:56.871034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c1786-71c7-408d-b8b1-f474b2ead7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:56.871585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c1786-71c7-408d-b8b1-f474b2ead7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:56.871681 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c1786-71c7-408d-b8b1-f474b2ead7dc admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:56.901079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1392/2771] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:56.903359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35425f14-18dc-4828-a074-58a242290c77 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:56.927450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35425f14-18dc-4828-a074-58a242290c77 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:56.947018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b602df3-c176-454e-8eb7-6d43780b798f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:56.948252 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602df3-c176-454e-8eb7-6d43780b798f admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:56.948697 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602df3-c176-454e-8eb7-6d43780b798f admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:56.949145 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602df3-c176-454e-8eb7-6d43780b798f admin admin] PATH_INFO: `/v3/projects/c371ae17178b4eb7a984382a9bffbd2f/users/a1ec5e0bc17d48159794362f7068af1e/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:57.000303 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b602df3-c176-454e-8eb7-6d43780b798f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c371ae17178b4eb7a984382a9bffbd2f, user_id=a1ec5e0bc17d48159794362f7068af1e, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:57.004830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b602df3-c176-454e-8eb7-6d43780b798f admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:57.024039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1380/2772] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:56 2022] PUT /identity/v3/projects/c371ae17178b4eb7a984382a9bffbd2f/users/a1ec5e0bc17d48159794362f7068af1e/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:57.029387 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe01ed-5059-4ca2-80ac-9d7a0fc33ed4 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:57.029754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe01ed-5059-4ca2-80ac-9d7a0fc33ed4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:57.030129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe01ed-5059-4ca2-80ac-9d7a0fc33ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:57.053105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-19fe01ed-5059-4ca2-80ac-9d7a0fc33ed4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:57.054778 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-19fe01ed-5059-4ca2-80ac-9d7a0fc33ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:57.143436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1393/2773] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:57.148392 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b602df3-c176-454e-8eb7-6d43780b798f admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:57.151339 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-3b602df3-c176-454e-8eb7-6d43780b798f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:57.238619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b602df3-c176-454e-8eb7-6d43780b798f admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:57.246567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f0b1e-064e-4516-a5d3-fca068b0c446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:57.247787 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f0b1e-064e-4516-a5d3-fca068b0c446 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:57.248395 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f0b1e-064e-4516-a5d3-fca068b0c446 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:57.248834 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f0b1e-064e-4516-a5d3-fca068b0c446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:57.271302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f0b1e-064e-4516-a5d3-fca068b0c446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:57.273860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f0b1e-064e-4516-a5d3-fca068b0c446 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:57.292474 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1381/2774] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:04:58.222339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b80ba-fb69-4897-bf0c-b6e0b2086baa None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:58.222976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b80ba-fb69-4897-bf0c-b6e0b2086baa None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:58.222976 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b80ba-fb69-4897-bf0c-b6e0b2086baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:58.253262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-200b80ba-fb69-4897-bf0c-b6e0b2086baa None None] MFA Rules not processed for user `a1ec5e0bc17d48159794362f7068af1e`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:58.254650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-200b80ba-fb69-4897-bf0c-b6e0b2086baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:58.389002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1394/2775] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:58.393812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f0b1e-064e-4516-a5d3-fca068b0c446 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:58.412951 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f0b1e-064e-4516-a5d3-fca068b0c446 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:58.420878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb614a-f902-46b3-a09b-5016bf1d59e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:58.422763 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb614a-f902-46b3-a09b-5016bf1d59e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:58.423277 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb614a-f902-46b3-a09b-5016bf1d59e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:58.423807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb614a-f902-46b3-a09b-5016bf1d59e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:58.424726 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb614a-f902-46b3-a09b-5016bf1d59e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:58.426966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb614a-f902-46b3-a09b-5016bf1d59e4 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:58.470171 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1382/2776] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:04:58 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:58.470865 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b80ba-fb69-4897-bf0c-b6e0b2086baa None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:58.489024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b80ba-fb69-4897-bf0c-b6e0b2086baa None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:58.582212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7c88b-6eb3-43ab-a849-7df7d837f678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:58.582212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7c88b-6eb3-43ab-a849-7df7d837f678 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:58.583563 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7c88b-6eb3-43ab-a849-7df7d837f678 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:58.584005 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7c88b-6eb3-43ab-a849-7df7d837f678 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:58.585350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7c88b-6eb3-43ab-a849-7df7d837f678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:58.589308 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7c88b-6eb3-43ab-a849-7df7d837f678 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:58.614874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1395/2777] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:04:58 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:04:58.618948 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb614a-f902-46b3-a09b-5016bf1d59e4 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:58.630376 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb614a-f902-46b3-a09b-5016bf1d59e4 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:58.636910 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9884ba-2bd4-48a4-b070-8f69e2a8560e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:58.637581 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9884ba-2bd4-48a4-b070-8f69e2a8560e admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:58.637860 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9884ba-2bd4-48a4-b070-8f69e2a8560e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:58.638138 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9884ba-2bd4-48a4-b070-8f69e2a8560e admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:58.639211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9884ba-2bd4-48a4-b070-8f69e2a8560e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:58.641278 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9884ba-2bd4-48a4-b070-8f69e2a8560e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:58.652336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1383/2778] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:04:58.656680 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7c88b-6eb3-43ab-a849-7df7d837f678 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:58.670959 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7c88b-6eb3-43ab-a849-7df7d837f678 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:58.676161 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714bdc2-54a6-4ca5-990a-9b0aca6f4b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:58.677079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714bdc2-54a6-4ca5-990a-9b0aca6f4b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:58.677362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714bdc2-54a6-4ca5-990a-9b0aca6f4b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:58.677637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714bdc2-54a6-4ca5-990a-9b0aca6f4b7d admin admin] PATH_INFO: `/v3/projects/05129c7bfbdf4a038f3c6a5cb400d2df/users/f10ecd1fb1264e8680359243b5c6f022/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:58.693872 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714bdc2-54a6-4ca5-990a-9b0aca6f4b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05129c7bfbdf4a038f3c6a5cb400d2df, user_id=f10ecd1fb1264e8680359243b5c6f022, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:58.696483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714bdc2-54a6-4ca5-990a-9b0aca6f4b7d admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:58.714187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1396/2779] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:58 2022] PUT /identity/v3/projects/05129c7bfbdf4a038f3c6a5cb400d2df/users/f10ecd1fb1264e8680359243b5c6f022/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:04:58.718332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9884ba-2bd4-48a4-b070-8f69e2a8560e admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:58.729097 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9884ba-2bd4-48a4-b070-8f69e2a8560e admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:58.793865 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2b51a-2eaf-41dd-abf1-864d2b9e80c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:58.794557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2b51a-2eaf-41dd-abf1-864d2b9e80c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:58.794900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2b51a-2eaf-41dd-abf1-864d2b9e80c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:58.795193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2b51a-2eaf-41dd-abf1-864d2b9e80c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:58.796611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2b51a-2eaf-41dd-abf1-864d2b9e80c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:58.798506 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2b51a-2eaf-41dd-abf1-864d2b9e80c5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:58.808857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1384/2780] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:04:58.813920 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714bdc2-54a6-4ca5-990a-9b0aca6f4b7d admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:58.824944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714bdc2-54a6-4ca5-990a-9b0aca6f4b7d admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:58.833393 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69767142-b52e-445b-9b7b-51488be88d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:58.834264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69767142-b52e-445b-9b7b-51488be88d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:58.834615 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69767142-b52e-445b-9b7b-51488be88d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:58.834994 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69767142-b52e-445b-9b7b-51488be88d7b admin admin] PATH_INFO: `/v3/projects/05129c7bfbdf4a038f3c6a5cb400d2df/users/f10ecd1fb1264e8680359243b5c6f022/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:58.837139 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69767142-b52e-445b-9b7b-51488be88d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=05129c7bfbdf4a038f3c6a5cb400d2df, user_id=f10ecd1fb1264e8680359243b5c6f022, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:58.839884 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69767142-b52e-445b-9b7b-51488be88d7b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:04:58.850677 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1397/2781] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:04:58 2022] PUT /identity/v3/projects/05129c7bfbdf4a038f3c6a5cb400d2df/users/f10ecd1fb1264e8680359243b5c6f022/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:04:59.635969 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bb38f-37fb-416e-97c8-471b905b9d43 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:59.637125 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bb38f-37fb-416e-97c8-471b905b9d43 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:59.637635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bb38f-37fb-416e-97c8-471b905b9d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:59.669003 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-8b7bb38f-37fb-416e-97c8-471b905b9d43 None None] MFA Rules not processed for user `f10ecd1fb1264e8680359243b5c6f022`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:04:59.670976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8b7bb38f-37fb-416e-97c8-471b905b9d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:59.763162 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1385/2782] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:04:59 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:04:59.776049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69767142-b52e-445b-9b7b-51488be88d7b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:04:59.778900 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-69767142-b52e-445b-9b7b-51488be88d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:04:59.878503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69767142-b52e-445b-9b7b-51488be88d7b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:04:59.883224 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a01662-0ff2-4bde-a72e-bb5b19979a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:04:59.884052 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01662-0ff2-4bde-a72e-bb5b19979a68 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:04:59.884576 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01662-0ff2-4bde-a72e-bb5b19979a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:04:59.884818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01662-0ff2-4bde-a72e-bb5b19979a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:04:59.985459 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a01662-0ff2-4bde-a72e-bb5b19979a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:04:59.987422 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a01662-0ff2-4bde-a72e-bb5b19979a68 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:00.007913 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1398/2783] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:04:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:01.063043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bb38f-37fb-416e-97c8-471b905b9d43 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:01.064504 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8b7bb38f-37fb-416e-97c8-471b905b9d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:01.129909 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bb38f-37fb-416e-97c8-471b905b9d43 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:01.134444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:01.135346 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:01.135626 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:01.135903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] PATH_INFO: `/v3/users/ec486e4fd685475bb903baac25cdf746` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:01.137139 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec486e4fd685475bb903baac25cdf746)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:01.139109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:01.186405 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec486e4fd685475bb903baac25cdf746'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:01.187203 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec486e4fd685475bb903baac25cdf746'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:01.200422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec486e4fd685475bb903baac25cdf746'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:01.201135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec486e4fd685475bb903baac25cdf746'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:01.208316 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec486e4fd685475bb903baac25cdf746'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:01.220779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1386/2784] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:05:01 2022] DELETE /identity/v3/users/ec486e4fd685475bb903baac25cdf746 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:01.225372 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8fc75-a7cb-421e-b093-cb295cc1c720 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:01.225856 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8fc75-a7cb-421e-b093-cb295cc1c720 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:01.226008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8fc75-a7cb-421e-b093-cb295cc1c720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:01.243087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-1cd8fc75-a7cb-421e-b093-cb295cc1c720 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:01.244219 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1cd8fc75-a7cb-421e-b093-cb295cc1c720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:01.303963 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1399/2785] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:01.313816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:01.315355 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:01.372719 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b204d3-02a3-486d-b721-dbe1e6ac7ef5 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:01.376888 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c202b-9907-427c-a55a-77da524d2cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:01.377522 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c202b-9907-427c-a55a-77da524d2cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:01.377797 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c202b-9907-427c-a55a-77da524d2cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:01.378066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c202b-9907-427c-a55a-77da524d2cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:01.387115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c202b-9907-427c-a55a-77da524d2cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:01.388548 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c202b-9907-427c-a55a-77da524d2cf6 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:01.399820 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1387/2786] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:01.485797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8fc75-a7cb-421e-b093-cb295cc1c720 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:01.487558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-1cd8fc75-a7cb-421e-b093-cb295cc1c720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:01.564002 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8fc75-a7cb-421e-b093-cb295cc1c720 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:01.569096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6ea36-324c-41ad-9500-5aa19c89eed7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:01.569892 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6ea36-324c-41ad-9500-5aa19c89eed7 service cinder] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:01.570175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6ea36-324c-41ad-9500-5aa19c89eed7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:01.570457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6ea36-324c-41ad-9500-5aa19c89eed7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:01.572294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-4fb6ea36-324c-41ad-9500-5aa19c89eed7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:01.647838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6ea36-324c-41ad-9500-5aa19c89eed7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:01.649782 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6ea36-324c-41ad-9500-5aa19c89eed7 service cinder] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:01.661028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1400/2787] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:01.915297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c202b-9907-427c-a55a-77da524d2cf6 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:01.915297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2b5c202b-9907-427c-a55a-77da524d2cf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:01.932859 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c202b-9907-427c-a55a-77da524d2cf6 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:01.941969 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:01.942972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:01.943345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:01.943673 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] PATH_INFO: `/v3/projects/9ec09ba22d804f168c25b95f1636d325` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:01.945309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec09ba22d804f168c25b95f1636d325)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:01.947969 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:02.002533 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec09ba22d804f168c25b95f1636d325', 'request_id': 'req-78962853-fc17-4dd1-b682-20dc431b5d7c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:02.009188 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec09ba22d804f168c25b95f1636d325', 'request_id': 'req-78962853-fc17-4dd1-b682-20dc431b5d7c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:02.015378 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec09ba22d804f168c25b95f1636d325', 'request_id': 'req-78962853-fc17-4dd1-b682-20dc431b5d7c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:02.028331 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] The token cache is being invalidate because project 9ec09ba22d804f168c25b95f1636d325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:05:02.028847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:02.029487 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-78962853-fc17-4dd1-b682-20dc431b5d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:02.031337 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1388/2788] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:05:01 2022] DELETE /identity/v3/projects/9ec09ba22d804f168c25b95f1636d325 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:02.036132 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c5bf9-2b2a-4070-86e5-ec27cfc0a8a2 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:02.036476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c5bf9-2b2a-4070-86e5-ec27cfc0a8a2 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:02.036755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c5bf9-2b2a-4070-86e5-ec27cfc0a8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:02.056169 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a11c5bf9-2b2a-4070-86e5-ec27cfc0a8a2 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:02.057386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a11c5bf9-2b2a-4070-86e5-ec27cfc0a8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:02.126143 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1401/2789] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:02.133546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9b1fd-956e-4686-9c4f-222af4cd0e23 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:02.134492 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9b1fd-956e-4686-9c4f-222af4cd0e23 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:02.134960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9b1fd-956e-4686-9c4f-222af4cd0e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:02.157773 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-55c9b1fd-956e-4686-9c4f-222af4cd0e23 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:02.159080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-55c9b1fd-956e-4686-9c4f-222af4cd0e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:02.167725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1389/2790] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:02.173059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c5bf9-2b2a-4070-86e5-ec27cfc0a8a2 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:02.183457 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c5bf9-2b2a-4070-86e5-ec27cfc0a8a2 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:02.187669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f6487-6fc7-47f0-9352-6e0554ed7cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:02.188335 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f6487-6fc7-47f0-9352-6e0554ed7cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:02.188616 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f6487-6fc7-47f0-9352-6e0554ed7cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:02.188897 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f6487-6fc7-47f0-9352-6e0554ed7cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:02.189687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f6487-6fc7-47f0-9352-6e0554ed7cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:05:02.189982 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f6487-6fc7-47f0-9352-6e0554ed7cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:02.191637 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f6487-6fc7-47f0-9352-6e0554ed7cc0 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:02.203832 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1402/2791] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:05:02.208456 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9b1fd-956e-4686-9c4f-222af4cd0e23 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:02.217620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9b1fd-956e-4686-9c4f-222af4cd0e23 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:02.222830 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2efbcc-c52a-48bd-8cf0-68a9f1edc185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:02.223467 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2efbcc-c52a-48bd-8cf0-68a9f1edc185 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:02.223744 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2efbcc-c52a-48bd-8cf0-68a9f1edc185 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:02.224018 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2efbcc-c52a-48bd-8cf0-68a9f1edc185 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:02.224798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2efbcc-c52a-48bd-8cf0-68a9f1edc185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:02.226622 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2efbcc-c52a-48bd-8cf0-68a9f1edc185 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:02.251851 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1390/2792] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:05:02 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:05:02.256244 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f6487-6fc7-47f0-9352-6e0554ed7cc0 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:02.266136 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f6487-6fc7-47f0-9352-6e0554ed7cc0 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:02.323311 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb492-c24f-432a-ba40-128570fbc91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:02.323989 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb492-c24f-432a-ba40-128570fbc91b admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:02.324269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb492-c24f-432a-ba40-128570fbc91b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:02.324544 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb492-c24f-432a-ba40-128570fbc91b admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:02.325245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cb492-c24f-432a-ba40-128570fbc91b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:02.327239 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cb492-c24f-432a-ba40-128570fbc91b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:02.350021 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1403/2793] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:05:02 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:05:02.354110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2efbcc-c52a-48bd-8cf0-68a9f1edc185 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:02.365588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2efbcc-c52a-48bd-8cf0-68a9f1edc185 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:02.370281 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6f02-124f-4fe6-b102-f76856277379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:02.370976 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6f02-124f-4fe6-b102-f76856277379 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:02.371262 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6f02-124f-4fe6-b102-f76856277379 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:02.371539 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6f02-124f-4fe6-b102-f76856277379 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:02.372256 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d6f02-124f-4fe6-b102-f76856277379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:02.373854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d6f02-124f-4fe6-b102-f76856277379 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:02.384307 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1391/2794] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:05:02.388269 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb492-c24f-432a-ba40-128570fbc91b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:02.398481 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb492-c24f-432a-ba40-128570fbc91b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:02.403186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a21710-5b59-4bff-b771-873fa6e16acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:02.403852 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21710-5b59-4bff-b771-873fa6e16acb admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:02.404133 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21710-5b59-4bff-b771-873fa6e16acb admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:02.404408 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21710-5b59-4bff-b771-873fa6e16acb admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:02.405128 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a21710-5b59-4bff-b771-873fa6e16acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:02.407954 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a21710-5b59-4bff-b771-873fa6e16acb admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:02.419170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1404/2795] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:05:02.424207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6f02-124f-4fe6-b102-f76856277379 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:02.433997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6f02-124f-4fe6-b102-f76856277379 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:02.438760 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922f577-303f-4d7f-94ad-de37951c6470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:02.439657 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922f577-303f-4d7f-94ad-de37951c6470 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:02.439972 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922f577-303f-4d7f-94ad-de37951c6470 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:02.440249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922f577-303f-4d7f-94ad-de37951c6470 admin admin] PATH_INFO: `/v3/projects/ed183118dd8c4a84a96cc1cdc92d3a7a/users/6ea439eafe224df0bd7fb46b6785e09b/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:02.455158 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922f577-303f-4d7f-94ad-de37951c6470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed183118dd8c4a84a96cc1cdc92d3a7a, user_id=6ea439eafe224df0bd7fb46b6785e09b, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:02.458328 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922f577-303f-4d7f-94ad-de37951c6470 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:02.559275 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1392/2796] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:05:02 2022] PUT /identity/v3/projects/ed183118dd8c4a84a96cc1cdc92d3a7a/users/6ea439eafe224df0bd7fb46b6785e09b/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:02.563013 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e672ef-15ea-4c81-b69e-8927008c5632 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:02.570034 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e672ef-15ea-4c81-b69e-8927008c5632 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:02.570464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e672ef-15ea-4c81-b69e-8927008c5632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:02.598728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-f5e672ef-15ea-4c81-b69e-8927008c5632 None None] MFA Rules not processed for user `6ea439eafe224df0bd7fb46b6785e09b`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:02.600060 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f5e672ef-15ea-4c81-b69e-8927008c5632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:02.663413 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1405/2797] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:02.674513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922f577-303f-4d7f-94ad-de37951c6470 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:02.676347 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9922f577-303f-4d7f-94ad-de37951c6470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:02.732555 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922f577-303f-4d7f-94ad-de37951c6470 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:02.736379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b7b2c-9c38-44ac-914a-170da6b8bf1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:02.737020 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b7b2c-9c38-44ac-914a-170da6b8bf1d service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:02.737297 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b7b2c-9c38-44ac-914a-170da6b8bf1d service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:02.737572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b7b2c-9c38-44ac-914a-170da6b8bf1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:02.746596 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b7b2c-9c38-44ac-914a-170da6b8bf1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:02.748093 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b7b2c-9c38-44ac-914a-170da6b8bf1d service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:02.757205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1393/2798] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:04.174243 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56ad16-db4d-4866-afef-d9b1153a60cc None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:04.174809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56ad16-db4d-4866-afef-d9b1153a60cc None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:04.174929 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56ad16-db4d-4866-afef-d9b1153a60cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:04.196406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-9d56ad16-db4d-4866-afef-d9b1153a60cc None None] MFA Rules not processed for user `6ea439eafe224df0bd7fb46b6785e09b`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:04.197688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9d56ad16-db4d-4866-afef-d9b1153a60cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:04.212775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1406/2799] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:14.816023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b7b2c-9c38-44ac-914a-170da6b8bf1d service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:14.826585 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b7b2c-9c38-44ac-914a-170da6b8bf1d service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:14.832163 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523262f5-221a-432d-869b-2d8345b9f543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:14.833327 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523262f5-221a-432d-869b-2d8345b9f543 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:14.834270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523262f5-221a-432d-869b-2d8345b9f543 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:14.834270 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523262f5-221a-432d-869b-2d8345b9f543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:14.836664 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-523262f5-221a-432d-869b-2d8345b9f543 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:14.901593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523262f5-221a-432d-869b-2d8345b9f543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:14.903215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523262f5-221a-432d-869b-2d8345b9f543 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:14.919349 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1394/2800] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:05:15.474534 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56ad16-db4d-4866-afef-d9b1153a60cc None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:15.476660 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-9d56ad16-db4d-4866-afef-d9b1153a60cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:15.549419 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56ad16-db4d-4866-afef-d9b1153a60cc None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:15.554025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:15.554922 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:15.555204 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:15.555479 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] PATH_INFO: `/v3/users/a1ec5e0bc17d48159794362f7068af1e` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:15.556684 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ec5e0bc17d48159794362f7068af1e)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:15.558657 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:15.607600 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ec5e0bc17d48159794362f7068af1e'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:15.620854 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ec5e0bc17d48159794362f7068af1e'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:15.631282 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ec5e0bc17d48159794362f7068af1e'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:15.632025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ec5e0bc17d48159794362f7068af1e'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:15.641059 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ec5e0bc17d48159794362f7068af1e'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:15.647264 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1407/2801] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:05:15 2022] DELETE /identity/v3/users/a1ec5e0bc17d48159794362f7068af1e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:15.833081 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523262f5-221a-432d-869b-2d8345b9f543 service cinder] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:15.845344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523262f5-221a-432d-869b-2d8345b9f543 service cinder] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:15.910216 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:15.911114 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:15.911386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:15.911654 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] PATH_INFO: `/v3/projects/c371ae17178b4eb7a984382a9bffbd2f` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:15.912854 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c371ae17178b4eb7a984382a9bffbd2f)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:15.914789 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:15.973350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c371ae17178b4eb7a984382a9bffbd2f', 'request_id': 'req-94f66fa9-7dd4-4062-8127-492c8bd86a9c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:15.979999 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c371ae17178b4eb7a984382a9bffbd2f', 'request_id': 'req-94f66fa9-7dd4-4062-8127-492c8bd86a9c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:15.986230 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c371ae17178b4eb7a984382a9bffbd2f', 'request_id': 'req-94f66fa9-7dd4-4062-8127-492c8bd86a9c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:15.998930 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] The token cache is being invalidate because project c371ae17178b4eb7a984382a9bffbd2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:05:15.999218 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:15.999856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.001423 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1395/2802] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:05:15 2022] DELETE /identity/v3/projects/c371ae17178b4eb7a984382a9bffbd2f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:16.005594 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:16.007115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:16.020961 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb34d2-5053-4e4f-a4b2-dc64fccb0285 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:16.076568 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:16.077452 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.078291 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.078569 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] PATH_INFO: `/v3/users/f10ecd1fb1264e8680359243b5c6f022` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:16.079838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=f10ecd1fb1264e8680359243b5c6f022)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:16.081769 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:16.129276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f10ecd1fb1264e8680359243b5c6f022'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.141366 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f10ecd1fb1264e8680359243b5c6f022'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.148177 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f10ecd1fb1264e8680359243b5c6f022'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.148918 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f10ecd1fb1264e8680359243b5c6f022'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.159935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e55bef20-880d-445e-a7f6-26bd8a415ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f10ecd1fb1264e8680359243b5c6f022'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.166736 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1408/2803] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:05:16 2022] DELETE /identity/v3/users/f10ecd1fb1264e8680359243b5c6f022 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:16.346986 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:16.348433 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:16.370002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f66fa9-7dd4-4062-8127-492c8bd86a9c admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:16.427926 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:16.428723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.429006 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.429284 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] PATH_INFO: `/v3/projects/05129c7bfbdf4a038f3c6a5cb400d2df` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:16.430529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05129c7bfbdf4a038f3c6a5cb400d2df)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:16.432517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:16.482080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05129c7bfbdf4a038f3c6a5cb400d2df', 'request_id': 'req-fb544e59-328c-465c-aa7d-57c7f368c24c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.488689 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05129c7bfbdf4a038f3c6a5cb400d2df', 'request_id': 'req-fb544e59-328c-465c-aa7d-57c7f368c24c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.494032 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05129c7bfbdf4a038f3c6a5cb400d2df', 'request_id': 'req-fb544e59-328c-465c-aa7d-57c7f368c24c'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.506914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] The token cache is being invalidate because project 05129c7bfbdf4a038f3c6a5cb400d2df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:05:16.507201 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.507833 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-fb544e59-328c-465c-aa7d-57c7f368c24c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:16.509359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1396/2804] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:05:16 2022] DELETE /identity/v3/projects/05129c7bfbdf4a038f3c6a5cb400d2df => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:16.515948 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d50e5-af7c-42ab-862d-3184b1ce08a4 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.516456 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d50e5-af7c-42ab-862d-3184b1ce08a4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.516570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d50e5-af7c-42ab-862d-3184b1ce08a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:16.536386 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-629d50e5-af7c-42ab-862d-3184b1ce08a4 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:16.537687 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-629d50e5-af7c-42ab-862d-3184b1ce08a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:16.605570 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1409/2805] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:16.615421 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563dc55-9f2c-46f1-8bd7-c2cd16272384 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.615715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563dc55-9f2c-46f1-8bd7-c2cd16272384 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.616135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563dc55-9f2c-46f1-8bd7-c2cd16272384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:16.638857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-a563dc55-9f2c-46f1-8bd7-c2cd16272384 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:16.640098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-a563dc55-9f2c-46f1-8bd7-c2cd16272384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:16.651208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1397/2806] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:16.655631 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d50e5-af7c-42ab-862d-3184b1ce08a4 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:16.666038 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d50e5-af7c-42ab-862d-3184b1ce08a4 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:16.670669 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeeb2b-995e-4ec8-b715-1085643adde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:16.673347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeeb2b-995e-4ec8-b715-1085643adde9 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.673347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeeb2b-995e-4ec8-b715-1085643adde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.673347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeeb2b-995e-4ec8-b715-1085643adde9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:16.673695 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aeeb2b-995e-4ec8-b715-1085643adde9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:05:16.673825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aeeb2b-995e-4ec8-b715-1085643adde9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:16.675443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aeeb2b-995e-4ec8-b715-1085643adde9 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:16.687596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1410/2807] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:05:16.692097 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563dc55-9f2c-46f1-8bd7-c2cd16272384 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:16.701779 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563dc55-9f2c-46f1-8bd7-c2cd16272384 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:16.706578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59583e2d-4147-4b83-82c5-c617551d1b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:16.707253 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59583e2d-4147-4b83-82c5-c617551d1b44 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.707536 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59583e2d-4147-4b83-82c5-c617551d1b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.707810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59583e2d-4147-4b83-82c5-c617551d1b44 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:16.708529 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59583e2d-4147-4b83-82c5-c617551d1b44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:16.710432 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59583e2d-4147-4b83-82c5-c617551d1b44 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:16.740717 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1398/2808] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:05:16 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:05:16.741914 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeeb2b-995e-4ec8-b715-1085643adde9 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:16.752049 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeeb2b-995e-4ec8-b715-1085643adde9 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:16.807658 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c7ce9-2f46-4285-8f5a-11eb7c7b8e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:16.808406 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c7ce9-2f46-4285-8f5a-11eb7c7b8e5c admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.808618 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c7ce9-2f46-4285-8f5a-11eb7c7b8e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.808919 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c7ce9-2f46-4285-8f5a-11eb7c7b8e5c admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:16.809592 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c7ce9-2f46-4285-8f5a-11eb7c7b8e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:16.811654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c7ce9-2f46-4285-8f5a-11eb7c7b8e5c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:16.834158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1411/2809] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:05:16 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:05:16.838243 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59583e2d-4147-4b83-82c5-c617551d1b44 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:16.851836 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59583e2d-4147-4b83-82c5-c617551d1b44 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:16.857086 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df8c71-bb16-41cd-afb0-473cea4b9212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:16.857765 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df8c71-bb16-41cd-afb0-473cea4b9212 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.858045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df8c71-bb16-41cd-afb0-473cea4b9212 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.858320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df8c71-bb16-41cd-afb0-473cea4b9212 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:16.859116 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df8c71-bb16-41cd-afb0-473cea4b9212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:16.860725 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df8c71-bb16-41cd-afb0-473cea4b9212 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:16.871411 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1399/2810] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:05:16.875436 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c7ce9-2f46-4285-8f5a-11eb7c7b8e5c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:16.885981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c7ce9-2f46-4285-8f5a-11eb7c7b8e5c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:16.891566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63651a43-85c4-429a-955e-ebec19537e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:16.892234 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63651a43-85c4-429a-955e-ebec19537e4a admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.892508 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63651a43-85c4-429a-955e-ebec19537e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.892788 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63651a43-85c4-429a-955e-ebec19537e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:16.893483 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63651a43-85c4-429a-955e-ebec19537e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:16.895285 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63651a43-85c4-429a-955e-ebec19537e4a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:16.906610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1412/2811] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:05:16.910672 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df8c71-bb16-41cd-afb0-473cea4b9212 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:16.921080 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df8c71-bb16-41cd-afb0-473cea4b9212 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:16.926545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bbac3-25d3-4128-a460-1d571dddbee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:16.927597 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bbac3-25d3-4128-a460-1d571dddbee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.927868 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bbac3-25d3-4128-a460-1d571dddbee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.928144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bbac3-25d3-4128-a460-1d571dddbee9 admin admin] PATH_INFO: `/v3/projects/8995bfdd77e44f24bc9ef6a52f90529b/users/64ede0bd1230418c8fb59b3c86e16233/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:16.943985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bbac3-25d3-4128-a460-1d571dddbee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8995bfdd77e44f24bc9ef6a52f90529b, user_id=64ede0bd1230418c8fb59b3c86e16233, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:16.946551 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bbac3-25d3-4128-a460-1d571dddbee9 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:16.960770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1400/2812] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:05:16 2022] PUT /identity/v3/projects/8995bfdd77e44f24bc9ef6a52f90529b/users/64ede0bd1230418c8fb59b3c86e16233/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:16.972124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbb164-2bb3-4b8e-972d-968fa91fa3cc None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:16.972543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbb164-2bb3-4b8e-972d-968fa91fa3cc None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:16.972865 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbb164-2bb3-4b8e-972d-968fa91fa3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:17.001295 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-63bbb164-2bb3-4b8e-972d-968fa91fa3cc None None] MFA Rules not processed for user `64ede0bd1230418c8fb59b3c86e16233`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:17.002528 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-63bbb164-2bb3-4b8e-972d-968fa91fa3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:17.069504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1413/2813] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:17.081624 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bbac3-25d3-4128-a460-1d571dddbee9 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:17.083350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f66bbac3-25d3-4128-a460-1d571dddbee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:17.139813 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bbac3-25d3-4128-a460-1d571dddbee9 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:17.144118 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d73356-08cf-4be4-a12e-8adc9aa0a13b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:17.144762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d73356-08cf-4be4-a12e-8adc9aa0a13b service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:17.145043 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d73356-08cf-4be4-a12e-8adc9aa0a13b service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:17.145315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d73356-08cf-4be4-a12e-8adc9aa0a13b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:17.153878 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d73356-08cf-4be4-a12e-8adc9aa0a13b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:17.155354 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d73356-08cf-4be4-a12e-8adc9aa0a13b service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:17.165932 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1401/2814] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:19.714087 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbb164-2bb3-4b8e-972d-968fa91fa3cc None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:19.788299 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbb164-2bb3-4b8e-972d-968fa91fa3cc None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:19.793080 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:19.794104 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:19.794391 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:19.794646 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] PATH_INFO: `/v3/users/64ede0bd1230418c8fb59b3c86e16233` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:19.795894 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ede0bd1230418c8fb59b3c86e16233)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:19.798558 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:19.861874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ede0bd1230418c8fb59b3c86e16233'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:19.883719 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ede0bd1230418c8fb59b3c86e16233'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:19.894398 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ede0bd1230418c8fb59b3c86e16233'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:19.896421 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ede0bd1230418c8fb59b3c86e16233'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:19.904160 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ede0bd1230418c8fb59b3c86e16233'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:19.917839 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1414/2815] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:05:19 2022] DELETE /identity/v3/users/64ede0bd1230418c8fb59b3c86e16233 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:19.919578 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b5408-3ead-47f3-8130-d593fe6dbe5f None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:19.920062 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b5408-3ead-47f3-8130-d593fe6dbe5f None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:19.920214 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b5408-3ead-47f3-8130-d593fe6dbe5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:19.944300 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-2f2b5408-3ead-47f3-8130-d593fe6dbe5f None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:19.945530 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2f2b5408-3ead-47f3-8130-d593fe6dbe5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:20.010122 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:20.011595 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:20.022302 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1402/2816] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:05:20.031963 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b5408-3ead-47f3-8130-d593fe6dbe5f None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:20.035013 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-2f2b5408-3ead-47f3-8130-d593fe6dbe5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:20.078596 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18b6c7-6b42-48ae-b3a1-635f1e09da4b admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:20.082881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942abc53-4082-4775-9598-f518d2a4e946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:20.083540 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942abc53-4082-4775-9598-f518d2a4e946 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:20.083825 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942abc53-4082-4775-9598-f518d2a4e946 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:20.084102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942abc53-4082-4775-9598-f518d2a4e946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:20.085951 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-942abc53-4082-4775-9598-f518d2a4e946 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:20.096546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b5408-3ead-47f3-8130-d593fe6dbe5f None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:20.101527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342b23c-12f6-484e-bbba-d5f2ed0a3a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:20.102102 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342b23c-12f6-484e-bbba-d5f2ed0a3a1b service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:20.102798 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342b23c-12f6-484e-bbba-d5f2ed0a3a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:20.103237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342b23c-12f6-484e-bbba-d5f2ed0a3a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:20.116087 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342b23c-12f6-484e-bbba-d5f2ed0a3a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:20.117560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342b23c-12f6-484e-bbba-d5f2ed0a3a1b service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:20.128546 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1403/2817] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:05:20.170465 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942abc53-4082-4775-9598-f518d2a4e946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:20.172820 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942abc53-4082-4775-9598-f518d2a4e946 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:20.184062 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1415/2818] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:05:20.509887 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342b23c-12f6-484e-bbba-d5f2ed0a3a1b service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:20.512101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e342b23c-12f6-484e-bbba-d5f2ed0a3a1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:20.537333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342b23c-12f6-484e-bbba-d5f2ed0a3a1b service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:20.545458 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:20.546499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:20.546828 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:20.547138 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] PATH_INFO: `/v3/projects/8995bfdd77e44f24bc9ef6a52f90529b` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:20.548696 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8995bfdd77e44f24bc9ef6a52f90529b)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:20.550593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:20.602000 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8995bfdd77e44f24bc9ef6a52f90529b', 'request_id': 'req-e0fa7947-bacf-470a-80e3-a40aa75abcdb'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:20.608711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8995bfdd77e44f24bc9ef6a52f90529b', 'request_id': 'req-e0fa7947-bacf-470a-80e3-a40aa75abcdb'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:20.615619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8995bfdd77e44f24bc9ef6a52f90529b', 'request_id': 'req-e0fa7947-bacf-470a-80e3-a40aa75abcdb'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:20.636172 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] The token cache is being invalidate because project 8995bfdd77e44f24bc9ef6a52f90529b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:05:20.636567 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:20.637430 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-e0fa7947-bacf-470a-80e3-a40aa75abcdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:20.639193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1404/2819] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:05:20 2022] DELETE /identity/v3/projects/8995bfdd77e44f24bc9ef6a52f90529b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:05:20.648381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e93864-fb40-4f85-8a90-e42b28ff67ec None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:20.649347 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e93864-fb40-4f85-8a90-e42b28ff67ec None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:20.649831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e93864-fb40-4f85-8a90-e42b28ff67ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:20.679209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-b6e93864-fb40-4f85-8a90-e42b28ff67ec None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:20.681245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-b6e93864-fb40-4f85-8a90-e42b28ff67ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:20.763037 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1416/2820] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:20.771525 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed8ab-3b79-413e-a317-3e2f83a75930 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:20.771816 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed8ab-3b79-413e-a317-3e2f83a75930 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:20.772130 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed8ab-3b79-413e-a317-3e2f83a75930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:20.803414 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-7e1ed8ab-3b79-413e-a317-3e2f83a75930 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:20.804740 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-7e1ed8ab-3b79-413e-a317-3e2f83a75930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:20.815537 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1405/2821] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:20.820564 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e93864-fb40-4f85-8a90-e42b28ff67ec None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:20.834802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e93864-fb40-4f85-8a90-e42b28ff67ec None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:20.842608 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ab2e3-f7ca-404f-b978-1b64c28f7970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:20.844198 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ab2e3-f7ca-404f-b978-1b64c28f7970 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:20.844485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ab2e3-f7ca-404f-b978-1b64c28f7970 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:20.844786 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ab2e3-f7ca-404f-b978-1b64c28f7970 admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:20.845499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ab2e3-f7ca-404f-b978-1b64c28f7970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:05:20.845867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ab2e3-f7ca-404f-b978-1b64c28f7970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:20.847651 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ab2e3-f7ca-404f-b978-1b64c28f7970 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:20.860186 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1417/2822] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:05:20.864995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ed8ab-3b79-413e-a317-3e2f83a75930 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:20.875187 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ed8ab-3b79-413e-a317-3e2f83a75930 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:20.880462 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be144e-0e60-4ac4-aeaf-2265bdfc3344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:20.881128 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be144e-0e60-4ac4-aeaf-2265bdfc3344 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:20.881448 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be144e-0e60-4ac4-aeaf-2265bdfc3344 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:20.881845 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be144e-0e60-4ac4-aeaf-2265bdfc3344 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:20.882621 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be144e-0e60-4ac4-aeaf-2265bdfc3344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:20.884566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be144e-0e60-4ac4-aeaf-2265bdfc3344 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:20.917097 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1406/2823] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:05:20 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:05:20.921566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ab2e3-f7ca-404f-b978-1b64c28f7970 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:20.932796 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ab2e3-f7ca-404f-b978-1b64c28f7970 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:21.004506 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1a2dc-f687-495f-bd07-905999a57f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:21.005192 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1a2dc-f687-495f-bd07-905999a57f57 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:21.005482 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1a2dc-f687-495f-bd07-905999a57f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:21.005762 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1a2dc-f687-495f-bd07-905999a57f57 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:21.006478 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1a2dc-f687-495f-bd07-905999a57f57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:21.008477 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1a2dc-f687-495f-bd07-905999a57f57 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:21.032176 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1418/2824] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:05:20 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:05:21.034731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be144e-0e60-4ac4-aeaf-2265bdfc3344 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:21.045495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be144e-0e60-4ac4-aeaf-2265bdfc3344 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:21.051618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d24dd-04af-4372-8b5c-ce8059c1ded3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:21.052377 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d24dd-04af-4372-8b5c-ce8059c1ded3 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:21.052729 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d24dd-04af-4372-8b5c-ce8059c1ded3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:21.053101 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d24dd-04af-4372-8b5c-ce8059c1ded3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:21.053903 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d24dd-04af-4372-8b5c-ce8059c1ded3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:21.055630 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d24dd-04af-4372-8b5c-ce8059c1ded3 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:21.067103 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1407/2825] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:05:21.071191 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1a2dc-f687-495f-bd07-905999a57f57 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:21.083091 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1a2dc-f687-495f-bd07-905999a57f57 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:21.088354 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe4d7f-527c-47af-ba6d-274d292bbd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:21.089025 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe4d7f-527c-47af-ba6d-274d292bbd3a admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:21.089336 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe4d7f-527c-47af-ba6d-274d292bbd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:21.089585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe4d7f-527c-47af-ba6d-274d292bbd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:21.090307 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe4d7f-527c-47af-ba6d-274d292bbd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:21.092115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe4d7f-527c-47af-ba6d-274d292bbd3a admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:21.103221 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1419/2826] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 14:05:21.107481 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d24dd-04af-4372-8b5c-ce8059c1ded3 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:21.117635 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d24dd-04af-4372-8b5c-ce8059c1ded3 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:21.122325 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04db5f1-5d95-4b3f-b439-99b71b9f16df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:21.123211 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04db5f1-5d95-4b3f-b439-99b71b9f16df admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:21.123486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04db5f1-5d95-4b3f-b439-99b71b9f16df admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:21.123752 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04db5f1-5d95-4b3f-b439-99b71b9f16df admin admin] PATH_INFO: `/v3/projects/07dc7904074a4edab0b334a71b8fa679/users/c469b7f312f748a4b107e329728b8b44/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:21.141924 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04db5f1-5d95-4b3f-b439-99b71b9f16df admin admin] RBAC: Authorizing `identity:create_grant(project_id=07dc7904074a4edab0b334a71b8fa679, user_id=c469b7f312f748a4b107e329728b8b44, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:21.146502 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04db5f1-5d95-4b3f-b439-99b71b9f16df admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:21.163615 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1408/2827] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:05:21 2022] PUT /identity/v3/projects/07dc7904074a4edab0b334a71b8fa679/users/c469b7f312f748a4b107e329728b8b44/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:21.175003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5204d-1965-4070-8a71-205933465109 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:21.175293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5204d-1965-4070-8a71-205933465109 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:21.175575 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5204d-1965-4070-8a71-205933465109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:21.204030 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-3af5204d-1965-4070-8a71-205933465109 None None] MFA Rules not processed for user `c469b7f312f748a4b107e329728b8b44`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:21.205512 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-3af5204d-1965-4070-8a71-205933465109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:21.277670 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1420/2828] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:21 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:05:21.292095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04db5f1-5d95-4b3f-b439-99b71b9f16df admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:21.294350 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c04db5f1-5d95-4b3f-b439-99b71b9f16df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:21.357461 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04db5f1-5d95-4b3f-b439-99b71b9f16df admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:21.362847 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441881c-003b-4bee-9942-54e12d166d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03fa33a68ff4567bc25c9611cd9ae8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:21.363619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441881c-003b-4bee-9942-54e12d166d04 service cinder] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:21.363989 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441881c-003b-4bee-9942-54e12d166d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:21.364359 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441881c-003b-4bee-9942-54e12d166d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:21.375556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441881c-003b-4bee-9942-54e12d166d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:21.377381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441881c-003b-4bee-9942-54e12d166d04 service cinder] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:21.387370 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1409/2829] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:21 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:22.954518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e159bc1-86eb-4dc8-b994-418201cdc4df None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:22.955070 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e159bc1-86eb-4dc8-b994-418201cdc4df None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:22.955174 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e159bc1-86eb-4dc8-b994-418201cdc4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:22.981179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-0e159bc1-86eb-4dc8-b994-418201cdc4df None None] MFA Rules not processed for user `a27e83f9f5fb4a0b99e7fac6ab90b3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:22.982486 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0e159bc1-86eb-4dc8-b994-418201cdc4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:23.082250 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1421/2830] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:05:23.273573 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2079f3a-6a48-429e-8fc9-c9a4f7724f99 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:23.275034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2079f3a-6a48-429e-8fc9-c9a4f7724f99 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:23.275509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2079f3a-6a48-429e-8fc9-c9a4f7724f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:23.299550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c2079f3a-6a48-429e-8fc9-c9a4f7724f99 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:23.301002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c2079f3a-6a48-429e-8fc9-c9a4f7724f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:23.406510 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1410/2831] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 14:05:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:23.423759 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e159bc1-86eb-4dc8-b994-418201cdc4df None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:23.425338 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-0e159bc1-86eb-4dc8-b994-418201cdc4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:23.556557 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e159bc1-86eb-4dc8-b994-418201cdc4df None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:23.568831 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2988c20-131a-4fb6-9be9-92a38b4fcc93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:23.571315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988c20-131a-4fb6-9be9-92a38b4fcc93 service swift] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:23.571315 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988c20-131a-4fb6-9be9-92a38b4fcc93 service swift] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:23.572158 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2988c20-131a-4fb6-9be9-92a38b4fcc93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:23.586979 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2988c20-131a-4fb6-9be9-92a38b4fcc93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:23.589008 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2988c20-131a-4fb6-9be9-92a38b4fcc93 service swift] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:23.603210 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1422/2832] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:05:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 13 14:05:23.609199 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2079f3a-6a48-429e-8fc9-c9a4f7724f99 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:23.612607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c2079f3a-6a48-429e-8fc9-c9a4f7724f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:23.703727 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2079f3a-6a48-429e-8fc9-c9a4f7724f99 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:23.709095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b0260-f1fe-4e30-882d-af54a085f898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:23.709871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b0260-f1fe-4e30-882d-af54a085f898 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:23.710149 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b0260-f1fe-4e30-882d-af54a085f898 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:23.710422 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b0260-f1fe-4e30-882d-af54a085f898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:23.714848 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-e01b0260-f1fe-4e30-882d-af54a085f898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:23.735056 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b0260-f1fe-4e30-882d-af54a085f898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:23.737249 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b0260-f1fe-4e30-882d-af54a085f898 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:23.749958 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1411/2833] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:24.317863 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d21c-c4fc-4b3d-8bf8-b195f70d17f4 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:24.318443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d21c-c4fc-4b3d-8bf8-b195f70d17f4 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:24.318443 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d21c-c4fc-4b3d-8bf8-b195f70d17f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:24.340927 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-46b4d21c-c4fc-4b3d-8bf8-b195f70d17f4 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:24.342339 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-46b4d21c-c4fc-4b3d-8bf8-b195f70d17f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:24.353775 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1423/2834] 10.222.0.79 () {58 vars in 967 bytes} [Fri May 13 14:05:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:24.373509 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b0260-f1fe-4e30-882d-af54a085f898 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:24.388493 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b0260-f1fe-4e30-882d-af54a085f898 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:24.395259 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788739c-1267-4c22-967a-a89f567409c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:24.395912 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788739c-1267-4c22-967a-a89f567409c1 service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:24.396189 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788739c-1267-4c22-967a-a89f567409c1 service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:24.396514 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788739c-1267-4c22-967a-a89f567409c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:24.405215 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788739c-1267-4c22-967a-a89f567409c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:24.406939 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788739c-1267-4c22-967a-a89f567409c1 service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:24.416512 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1412/2835] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:05:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 14:05:24.680826 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0d24-a2f8-4d44-b3a4-953415526be3 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:24.681147 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0d24-a2f8-4d44-b3a4-953415526be3 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:24.681426 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0d24-a2f8-4d44-b3a4-953415526be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:24.704179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-635a0d24-a2f8-4d44-b3a4-953415526be3 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:24.704970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-635a0d24-a2f8-4d44-b3a4-953415526be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:24.717626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1424/2836] 10.222.0.79 () {58 vars in 1009 bytes} [Fri May 13 14:05:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:24.733269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788739c-1267-4c22-967a-a89f567409c1 service swift] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:24.765476 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788739c-1267-4c22-967a-a89f567409c1 service swift] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:24.772257 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b34f8-d26b-468f-a018-662795356d1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:24.772995 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b34f8-d26b-468f-a018-662795356d1d service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:24.773352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b34f8-d26b-468f-a018-662795356d1d service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:24.773703 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b34f8-d26b-468f-a018-662795356d1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:24.784352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b34f8-d26b-468f-a018-662795356d1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:24.786066 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b34f8-d26b-468f-a018-662795356d1d service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:24.809311 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1413/2837] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:05:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 13 14:05:25.322368 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18085ce5-9ab2-477f-a47c-9c46d5708ec1 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:25.323042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18085ce5-9ab2-477f-a47c-9c46d5708ec1 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:25.323042 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18085ce5-9ab2-477f-a47c-9c46d5708ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:25.355093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-18085ce5-9ab2-477f-a47c-9c46d5708ec1 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:25.357069 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-18085ce5-9ab2-477f-a47c-9c46d5708ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:25.367632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1425/2838] 10.222.0.79 () {58 vars in 967 bytes} [Fri May 13 14:05:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:25.383323 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b34f8-d26b-468f-a018-662795356d1d service swift] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:25.396969 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b34f8-d26b-468f-a018-662795356d1d service swift] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:25.403046 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95285b90-7940-498b-aa94-680a0fbab678 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:25.416386 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95285b90-7940-498b-aa94-680a0fbab678 service swift] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:25.416677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95285b90-7940-498b-aa94-680a0fbab678 service swift] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:25.416957 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95285b90-7940-498b-aa94-680a0fbab678 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:25.436980 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95285b90-7940-498b-aa94-680a0fbab678 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:25.439147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95285b90-7940-498b-aa94-680a0fbab678 service swift] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:25.448389 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1414/2839] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:05:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 14:05:27.929917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18085ce5-9ab2-477f-a47c-9c46d5708ec1 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:27.931449 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-18085ce5-9ab2-477f-a47c-9c46d5708ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:28.020078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18085ce5-9ab2-477f-a47c-9c46d5708ec1 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:28.025844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:28.026907 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:28.027312 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:28.027698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] PATH_INFO: `/v3/users/6ea439eafe224df0bd7fb46b6785e09b` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:28.029209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ea439eafe224df0bd7fb46b6785e09b)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:28.031527 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:28.107668 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ea439eafe224df0bd7fb46b6785e09b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:28.121932 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ea439eafe224df0bd7fb46b6785e09b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:28.130739 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ea439eafe224df0bd7fb46b6785e09b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:28.131493 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ea439eafe224df0bd7fb46b6785e09b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:28.141635 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ea439eafe224df0bd7fb46b6785e09b'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:28.149542 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1426/2840] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:05:27 2022] DELETE /identity/v3/users/6ea439eafe224df0bd7fb46b6785e09b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:28.154404 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75c49e-6304-45a3-b922-951fbc512795 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:28.154960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75c49e-6304-45a3-b922-951fbc512795 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:28.155060 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75c49e-6304-45a3-b922-951fbc512795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:28.174046 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-cc75c49e-6304-45a3-b922-951fbc512795 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:28.175309 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-cc75c49e-6304-45a3-b922-951fbc512795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:28.243891 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1415/2841] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:28.254610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:28.256102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:28.334144 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0d716-18ee-4d1c-8d03-37347e918507 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:28.338664 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d46ae6-e137-4772-8aa6-6212d0e3b145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:28.339345 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d46ae6-e137-4772-8aa6-6212d0e3b145 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:28.339620 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d46ae6-e137-4772-8aa6-6212d0e3b145 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:28.339891 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d46ae6-e137-4772-8aa6-6212d0e3b145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:28.350118 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d46ae6-e137-4772-8aa6-6212d0e3b145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:28.351593 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d46ae6-e137-4772-8aa6-6212d0e3b145 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:28.361898 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1427/2842] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:28.554129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75c49e-6304-45a3-b922-951fbc512795 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:28.555706 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-cc75c49e-6304-45a3-b922-951fbc512795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:28.568078 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75c49e-6304-45a3-b922-951fbc512795 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:28.573441 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:28.574640 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:28.575048 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:28.575324 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] PATH_INFO: `/v3/projects/ed183118dd8c4a84a96cc1cdc92d3a7a` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:28.576586 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed183118dd8c4a84a96cc1cdc92d3a7a)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:28.578766 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:28.644674 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed183118dd8c4a84a96cc1cdc92d3a7a', 'request_id': 'req-9f93ac09-2cde-4064-b146-39279683f981'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:28.652715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed183118dd8c4a84a96cc1cdc92d3a7a', 'request_id': 'req-9f93ac09-2cde-4064-b146-39279683f981'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:28.659990 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed183118dd8c4a84a96cc1cdc92d3a7a', 'request_id': 'req-9f93ac09-2cde-4064-b146-39279683f981'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:28.674517 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] The token cache is being invalidate because project ed183118dd8c4a84a96cc1cdc92d3a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:05:28.675205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:28.676027 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-9f93ac09-2cde-4064-b146-39279683f981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:05:28.678633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1416/2843] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:05:28 2022] DELETE /identity/v3/projects/ed183118dd8c4a84a96cc1cdc92d3a7a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:28.683579 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7ac6c-eaee-4e29-96e2-1c3125034fe8 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:28.683874 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7ac6c-eaee-4e29-96e2-1c3125034fe8 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:28.684154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7ac6c-eaee-4e29-96e2-1c3125034fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:28.705012 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-62c7ac6c-eaee-4e29-96e2-1c3125034fe8 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:28.708636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-62c7ac6c-eaee-4e29-96e2-1c3125034fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:28.793035 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1428/2844] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:28.802074 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92511b-24d6-4b84-8fa3-64a27a273523 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:28.802381 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92511b-24d6-4b84-8fa3-64a27a273523 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:28.802655 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92511b-24d6-4b84-8fa3-64a27a273523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:28.823681 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-1f92511b-24d6-4b84-8fa3-64a27a273523 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:28.824812 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1f92511b-24d6-4b84-8fa3-64a27a273523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:28.833294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1417/2845] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:28.840078 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7ac6c-eaee-4e29-96e2-1c3125034fe8 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:28.850522 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7ac6c-eaee-4e29-96e2-1c3125034fe8 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:28.855209 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c24b09-e487-47a0-b93c-ae77bf0132bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:28.855883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c24b09-e487-47a0-b93c-ae77bf0132bd admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:28.856164 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c24b09-e487-47a0-b93c-ae77bf0132bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:28.856485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c24b09-e487-47a0-b93c-ae77bf0132bd admin admin] PATH_INFO: `/v3/domains` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:28.857156 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c24b09-e487-47a0-b93c-ae77bf0132bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 14:05:28.857451 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c24b09-e487-47a0-b93c-ae77bf0132bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:28.859772 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c24b09-e487-47a0-b93c-ae77bf0132bd admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:28.878688 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1429/2846] 10.222.0.79 () {58 vars in 1152 bytes} [Fri May 13 14:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 14:05:28.880445 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92511b-24d6-4b84-8fa3-64a27a273523 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:28.891550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92511b-24d6-4b84-8fa3-64a27a273523 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:28.896762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c1e42-fef5-49ec-a755-2131e450af58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:28.897673 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c1e42-fef5-49ec-a755-2131e450af58 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:28.898008 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c1e42-fef5-49ec-a755-2131e450af58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:28.898424 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c1e42-fef5-49ec-a755-2131e450af58 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:28.899229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c1e42-fef5-49ec-a755-2131e450af58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:28.901177 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c1e42-fef5-49ec-a755-2131e450af58 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:28.931742 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1418/2847] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:05:28 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:05:28.932850 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c24b09-e487-47a0-b93c-ae77bf0132bd admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:28.946537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c24b09-e487-47a0-b93c-ae77bf0132bd admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:29.022597 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c732b4-f45d-4cab-bafa-dd4b380f5075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:29.023381 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c732b4-f45d-4cab-bafa-dd4b380f5075 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:29.023550 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c732b4-f45d-4cab-bafa-dd4b380f5075 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:29.023823 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c732b4-f45d-4cab-bafa-dd4b380f5075 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:29.024626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c732b4-f45d-4cab-bafa-dd4b380f5075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:29.026673 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c732b4-f45d-4cab-bafa-dd4b380f5075 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:29.050024 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1430/2848] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:05:28 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 14:05:29.054463 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c1e42-fef5-49ec-a755-2131e450af58 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:29.066034 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c1e42-fef5-49ec-a755-2131e450af58 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:29.070807 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6afa6-5ed3-433f-98b8-e1ddf96fb154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:29.071490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6afa6-5ed3-433f-98b8-e1ddf96fb154 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:29.071763 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6afa6-5ed3-433f-98b8-e1ddf96fb154 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:29.072036 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6afa6-5ed3-433f-98b8-e1ddf96fb154 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:29.072770 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6afa6-5ed3-433f-98b8-e1ddf96fb154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:29.074419 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6afa6-5ed3-433f-98b8-e1ddf96fb154 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:29.089207 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1419/2849] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:05:29.093944 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c732b4-f45d-4cab-bafa-dd4b380f5075 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:29.105028 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c732b4-f45d-4cab-bafa-dd4b380f5075 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:29.111159 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39994ab-b1c1-42e7-86fe-e1db1dc08597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:29.111718 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39994ab-b1c1-42e7-86fe-e1db1dc08597 admin admin] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:29.112019 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39994ab-b1c1-42e7-86fe-e1db1dc08597 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:29.112266 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39994ab-b1c1-42e7-86fe-e1db1dc08597 admin admin] PATH_INFO: `/v3/roles` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:29.113003 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39994ab-b1c1-42e7-86fe-e1db1dc08597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:29.114653 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39994ab-b1c1-42e7-86fe-e1db1dc08597 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:29.126765 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1431/2850] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:05:29.131098 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6afa6-5ed3-433f-98b8-e1ddf96fb154 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:29.141455 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6afa6-5ed3-433f-98b8-e1ddf96fb154 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:29.146095 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086c7df-22ee-4683-b3c0-da670d672abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:29.146955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086c7df-22ee-4683-b3c0-da670d672abd admin admin] REQUEST_METHOD: `PUT` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:29.147234 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086c7df-22ee-4683-b3c0-da670d672abd admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:29.147677 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086c7df-22ee-4683-b3c0-da670d672abd admin admin] PATH_INFO: `/v3/projects/54b153696d2b445098e70dce631b13b6/users/4ac7988aa0e74fc6b56269d975270d48/roles/90868071f00340ae853f1188777a2244` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:29.164205 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086c7df-22ee-4683-b3c0-da670d672abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b153696d2b445098e70dce631b13b6, user_id=4ac7988aa0e74fc6b56269d975270d48, role_id=90868071f00340ae853f1188777a2244)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:29.166882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086c7df-22ee-4683-b3c0-da670d672abd admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:29.181863 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1420/2851] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:05:29 2022] PUT /identity/v3/projects/54b153696d2b445098e70dce631b13b6/users/4ac7988aa0e74fc6b56269d975270d48/roles/90868071f00340ae853f1188777a2244 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:29.187183 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767be0e-47a9-4172-a02c-68fbe819f7b1 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:29.187626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767be0e-47a9-4172-a02c-68fbe819f7b1 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:29.187924 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767be0e-47a9-4172-a02c-68fbe819f7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:29.206867 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a767be0e-47a9-4172-a02c-68fbe819f7b1 None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:29.208131 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a767be0e-47a9-4172-a02c-68fbe819f7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:29.267102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1432/2852] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:29.277014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086c7df-22ee-4683-b3c0-da670d672abd admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:29.278786 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c086c7df-22ee-4683-b3c0-da670d672abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:29.337698 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086c7df-22ee-4683-b3c0-da670d672abd admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:29.341634 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc283a-1808-4314-b10b-e7ebfcedf0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:29.342274 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc283a-1808-4314-b10b-e7ebfcedf0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:29.342556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc283a-1808-4314-b10b-e7ebfcedf0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:29.342869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc283a-1808-4314-b10b-e7ebfcedf0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:29.353499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc283a-1808-4314-b10b-e7ebfcedf0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:29.355070 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc283a-1808-4314-b10b-e7ebfcedf0c8 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:29.366028 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1421/2853] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:33.735889 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c7c2-639b-409f-b71f-4a93c31931e6 None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:33.736675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c7c2-639b-409f-b71f-4a93c31931e6 None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:33.736675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c7c2-639b-409f-b71f-4a93c31931e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:33.765547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-c2d4c7c2-639b-409f-b71f-4a93c31931e6 None None] MFA Rules not processed for user `4ac7988aa0e74fc6b56269d975270d48`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:33.767124 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c2d4c7c2-639b-409f-b71f-4a93c31931e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:33.852097 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1433/2854] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 14:05:33.856623 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc283a-1808-4314-b10b-e7ebfcedf0c8 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:33.874061 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc283a-1808-4314-b10b-e7ebfcedf0c8 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:33.878905 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edd3df-6ea2-4f50-9952-c393fb7fb6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:33.879711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd3df-6ea2-4f50-9952-c393fb7fb6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:33.879998 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd3df-6ea2-4f50-9952-c393fb7fb6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:33.880269 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd3df-6ea2-4f50-9952-c393fb7fb6c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:33.880962 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edd3df-6ea2-4f50-9952-c393fb7fb6c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:33.882981 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edd3df-6ea2-4f50-9952-c393fb7fb6c5 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:33.914466 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1422/2855] 10.222.0.79 () {60 vars in 1152 bytes} [Fri May 13 14:05:33 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:05:33.915818 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4c7c2-639b-409f-b71f-4a93c31931e6 None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:33.928046 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4c7c2-639b-409f-b71f-4a93c31931e6 None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:33.992981 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8013ef-245b-40d3-a9cf-f55a7da1e0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:33.993656 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8013ef-245b-40d3-a9cf-f55a7da1e0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:33.993934 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8013ef-245b-40d3-a9cf-f55a7da1e0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:33.994212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8013ef-245b-40d3-a9cf-f55a7da1e0b5 admin admin] PATH_INFO: `/v3/users` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:33.994930 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8013ef-245b-40d3-a9cf-f55a7da1e0b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:33.996860 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8013ef-245b-40d3-a9cf-f55a7da1e0b5 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:34.019107 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1434/2856] 10.222.0.79 () {60 vars in 1143 bytes} [Fri May 13 14:05:33 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 14:05:34.023454 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edd3df-6ea2-4f50-9952-c393fb7fb6c5 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:34.035135 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edd3df-6ea2-4f50-9952-c393fb7fb6c5 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:34.040396 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19307f8-5ddc-487c-abf3-430995d40b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:34.041069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19307f8-5ddc-487c-abf3-430995d40b72 admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:34.041345 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19307f8-5ddc-487c-abf3-430995d40b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:34.041618 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19307f8-5ddc-487c-abf3-430995d40b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:34.042333 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19307f8-5ddc-487c-abf3-430995d40b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:34.044129 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19307f8-5ddc-487c-abf3-430995d40b72 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:34.056557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1423/2857] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:05:34.060881 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8013ef-245b-40d3-a9cf-f55a7da1e0b5 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:34.071817 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8013ef-245b-40d3-a9cf-f55a7da1e0b5 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:34.079640 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474f6de-958e-4d74-9d82-d6d940fca8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:34.080555 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474f6de-958e-4d74-9d82-d6d940fca8c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:34.080838 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474f6de-958e-4d74-9d82-d6d940fca8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:34.081117 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474f6de-958e-4d74-9d82-d6d940fca8c6 admin admin] PATH_INFO: `/v3/projects/25aece37f16d46259f10198b17d91fa1/users/029e07de814546e59202df728df9a823/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:34.097562 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474f6de-958e-4d74-9d82-d6d940fca8c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25aece37f16d46259f10198b17d91fa1, user_id=029e07de814546e59202df728df9a823, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:34.100318 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474f6de-958e-4d74-9d82-d6d940fca8c6 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:34.115997 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1435/2858] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:05:34 2022] PUT /identity/v3/projects/25aece37f16d46259f10198b17d91fa1/users/029e07de814546e59202df728df9a823/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:34.120002 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19307f8-5ddc-487c-abf3-430995d40b72 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:34.130501 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19307f8-5ddc-487c-abf3-430995d40b72 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:34.196228 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5650153-a307-4273-82ad-e914f0be671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:34.196900 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5650153-a307-4273-82ad-e914f0be671e admin admin] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:34.197176 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5650153-a307-4273-82ad-e914f0be671e admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:34.197444 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5650153-a307-4273-82ad-e914f0be671e admin admin] PATH_INFO: `/v3/roles` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:34.198268 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5650153-a307-4273-82ad-e914f0be671e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:34.201021 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5650153-a307-4273-82ad-e914f0be671e admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:34.213045 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1424/2859] 10.222.0.79 () {58 vars in 1121 bytes} [Fri May 13 14:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 14:05:34.217099 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474f6de-958e-4d74-9d82-d6d940fca8c6 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:34.229115 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474f6de-958e-4d74-9d82-d6d940fca8c6 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:34.236010 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da1ed0-949d-4fc8-8d4c-4a2fecb774c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:34.237140 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1ed0-949d-4fc8-8d4c-4a2fecb774c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:34.237518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1ed0-949d-4fc8-8d4c-4a2fecb774c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:34.238050 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1ed0-949d-4fc8-8d4c-4a2fecb774c3 admin admin] PATH_INFO: `/v3/projects/25aece37f16d46259f10198b17d91fa1/users/029e07de814546e59202df728df9a823/roles/4416a11ddf5d4ed194c121f0e9ebf9b3` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:34.240623 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da1ed0-949d-4fc8-8d4c-4a2fecb774c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25aece37f16d46259f10198b17d91fa1, user_id=029e07de814546e59202df728df9a823, role_id=4416a11ddf5d4ed194c121f0e9ebf9b3)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:34.243425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da1ed0-949d-4fc8-8d4c-4a2fecb774c3 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:34.257745 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1436/2860] 10.222.0.79 () {60 vars in 1482 bytes} [Fri May 13 14:05:34 2022] PUT /identity/v3/projects/25aece37f16d46259f10198b17d91fa1/users/029e07de814546e59202df728df9a823/roles/4416a11ddf5d4ed194c121f0e9ebf9b3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:05:39.917069 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23713-b8ab-4e7a-8a19-5d0414278822 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:39.917593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23713-b8ab-4e7a-8a19-5d0414278822 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:39.917593 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23713-b8ab-4e7a-8a19-5d0414278822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:39.954882 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-28d23713-b8ab-4e7a-8a19-5d0414278822 None None] MFA Rules not processed for user `029e07de814546e59202df728df9a823`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:39.956372 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-28d23713-b8ab-4e7a-8a19-5d0414278822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:40.056588 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1425/2861] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:40.071590 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da1ed0-949d-4fc8-8d4c-4a2fecb774c3 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:40.073585 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-c0da1ed0-949d-4fc8-8d4c-4a2fecb774c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:40.158499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da1ed0-949d-4fc8-8d4c-4a2fecb774c3 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:40.164571 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447cc8e-c043-44b2-b6fb-93673d6ffeb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:40.165306 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447cc8e-c043-44b2-b6fb-93673d6ffeb8 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:40.165598 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447cc8e-c043-44b2-b6fb-93673d6ffeb8 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:40.165883 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447cc8e-c043-44b2-b6fb-93673d6ffeb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:40.239212 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447cc8e-c043-44b2-b6fb-93673d6ffeb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:40.245098 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447cc8e-c043-44b2-b6fb-93673d6ffeb8 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:40.256377 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1437/2862] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:41.628412 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5a3a3-eb96-45c5-ad71-e782c5f8355b None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:41.628955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5a3a3-eb96-45c5-ad71-e782c5f8355b None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:41.628955 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5a3a3-eb96-45c5-ad71-e782c5f8355b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:41.648450 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-56d5a3a3-eb96-45c5-ad71-e782c5f8355b None None] MFA Rules not processed for user `4ac7988aa0e74fc6b56269d975270d48`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:05:41.649675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-56d5a3a3-eb96-45c5-ad71-e782c5f8355b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:41.660750 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1426/2863] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:05:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:05:42.749610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447cc8e-c043-44b2-b6fb-93673d6ffeb8 service nova] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:42.816935 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447cc8e-c043-44b2-b6fb-93673d6ffeb8 service nova] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:42.821794 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68f27-236e-4d17-8f77-2315a017d054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:42.822496 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68f27-236e-4d17-8f77-2315a017d054 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:42.822873 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68f27-236e-4d17-8f77-2315a017d054 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:42.823170 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68f27-236e-4d17-8f77-2315a017d054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:42.826357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-56d68f27-236e-4d17-8f77-2315a017d054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:42.913044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d68f27-236e-4d17-8f77-2315a017d054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:42.913044 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d68f27-236e-4d17-8f77-2315a017d054 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:42.923750 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1438/2864] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:54.782861 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5a3a3-eb96-45c5-ad71-e782c5f8355b None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:54.785320 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-56d5a3a3-eb96-45c5-ad71-e782c5f8355b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:54.809397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5a3a3-eb96-45c5-ad71-e782c5f8355b None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:54.815447 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ec4cc-8033-43ed-a403-71040f88b49b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:54.816110 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ec4cc-8033-43ed-a403-71040f88b49b service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:54.816782 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ec4cc-8033-43ed-a403-71040f88b49b service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:54.817068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ec4cc-8033-43ed-a403-71040f88b49b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:54.818959 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5e4ec4cc-8033-43ed-a403-71040f88b49b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:54.834571 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ec4cc-8033-43ed-a403-71040f88b49b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:54.836072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ec4cc-8033-43ed-a403-71040f88b49b service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:54.849495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1427/2865] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:05:56.166862 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68f27-236e-4d17-8f77-2315a017d054 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:05:56.177503 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68f27-236e-4d17-8f77-2315a017d054 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:05:56.182289 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096f995-0aa0-4afb-a369-a40bb2d68223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:05:56.183121 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096f995-0aa0-4afb-a369-a40bb2d68223 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:05:56.183425 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096f995-0aa0-4afb-a369-a40bb2d68223 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:05:56.183916 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096f995-0aa0-4afb-a369-a40bb2d68223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:05:56.187109 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f096f995-0aa0-4afb-a369-a40bb2d68223 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:05:56.203155 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096f995-0aa0-4afb-a369-a40bb2d68223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:05:56.204626 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096f995-0aa0-4afb-a369-a40bb2d68223 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:05:56.216712 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1439/2866] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:05:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:06:04.301556 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ec4cc-8033-43ed-a403-71040f88b49b service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:06:04.303193 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5e4ec4cc-8033-43ed-a403-71040f88b49b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:04.374872 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ec4cc-8033-43ed-a403-71040f88b49b service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:06:04.378271 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:06:04.379375 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:06:04.379659 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:06:04.379983 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] PATH_INFO: `/v3/users/c469b7f312f748a4b107e329728b8b44` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:06:04.381323 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c469b7f312f748a4b107e329728b8b44)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:06:04.383336 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:06:04.431109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c469b7f312f748a4b107e329728b8b44'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:06:04.431772 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c469b7f312f748a4b107e329728b8b44'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:06:04.444082 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c469b7f312f748a4b107e329728b8b44'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:06:04.444822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c469b7f312f748a4b107e329728b8b44'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:06:04.451479 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c469b7f312f748a4b107e329728b8b44'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:06:04.463987 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1428/2867] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:06:04 2022] DELETE /identity/v3/users/c469b7f312f748a4b107e329728b8b44 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:06:04.468678 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8927a75-bdcb-419c-9ac8-0b090e9e3a5c None None] REQUEST_METHOD: `POST` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:06:04.470213 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8927a75-bdcb-419c-9ac8-0b090e9e3a5c None None] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:06:04.470499 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8927a75-bdcb-419c-9ac8-0b090e9e3a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:06:04.488573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.auth.core [None req-a8927a75-bdcb-419c-9ac8-0b090e9e3a5c None None] MFA Rules not processed for user `a0034eb1f585466fae466bc41ea6f933`. Rule list: `[]` (Enabled: `True`). {{(pid=86363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:06:04.489797 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a8927a75-bdcb-419c-9ac8-0b090e9e3a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:04.556622 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1440/2868] 10.222.0.79 () {58 vars in 957 bytes} [Fri May 13 14:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:06:04.566773 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:06:04.568241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:04.632966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7b274-1751-4094-a4dc-7abc8493c00c admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:06:04.638342 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17778c8-45e4-478f-9f4c-b8b643e22b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:06:04.639486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17778c8-45e4-478f-9f4c-b8b643e22b2e service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:06:04.639620 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17778c8-45e4-478f-9f4c-b8b643e22b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:06:04.639881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17778c8-45e4-478f-9f4c-b8b643e22b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:06:04.650310 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17778c8-45e4-478f-9f4c-b8b643e22b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:06:04.651822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17778c8-45e4-478f-9f4c-b8b643e22b2e service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:06:04.665306 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1429/2869] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:06:05.067844 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8927a75-bdcb-419c-9ac8-0b090e9e3a5c None None] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:06:05.069467 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-a8927a75-bdcb-419c-9ac8-0b090e9e3a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:05.085092 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8927a75-bdcb-419c-9ac8-0b090e9e3a5c None None] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:06:05.090273 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:06:05.091228 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:06:05.091614 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:06:05.091946 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] PATH_INFO: `/v3/projects/07dc7904074a4edab0b334a71b8fa679` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:06:05.094440 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07dc7904074a4edab0b334a71b8fa679)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:06:05.096732 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:06:05.152755 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07dc7904074a4edab0b334a71b8fa679', 'request_id': 'req-608855d0-9bd8-4c07-b10d-5ea5fe936457'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:06:05.160102 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07dc7904074a4edab0b334a71b8fa679', 'request_id': 'req-608855d0-9bd8-4c07-b10d-5ea5fe936457'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:06:05.172362 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07dc7904074a4edab0b334a71b8fa679', 'request_id': 'req-608855d0-9bd8-4c07-b10d-5ea5fe936457'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:06:05.192323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] The token cache is being invalidate because project 07dc7904074a4edab0b334a71b8fa679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:06:05.192675 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:06:05.193410 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:06:05.195432 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1441/2870] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:06:05 2022] DELETE /identity/v3/projects/07dc7904074a4edab0b334a71b8fa679 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:06:18.853531 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17778c8-45e4-478f-9f4c-b8b643e22b2e service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:06:18.862094 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17778c8-45e4-478f-9f4c-b8b643e22b2e service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:06:18.909109 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cc0c8-8b9b-4c25-971b-11ffb38fae9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:06:18.909746 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cc0c8-8b9b-4c25-971b-11ffb38fae9e service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:06:18.910023 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cc0c8-8b9b-4c25-971b-11ffb38fae9e service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:06:18.910294 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cc0c8-8b9b-4c25-971b-11ffb38fae9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:06:18.912113 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f82cc0c8-8b9b-4c25-971b-11ffb38fae9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:18.923009 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cc0c8-8b9b-4c25-971b-11ffb38fae9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:06:18.924633 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cc0c8-8b9b-4c25-971b-11ffb38fae9e service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:06:18.932857 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1430/2871] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:06:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:06:20.893655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:06:20.895293 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:20.907628 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608855d0-9bd8-4c07-b10d-5ea5fe936457 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:06:20.911437 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee35e5-f9df-45a4-a2d6-c73d47104ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:06:20.912093 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee35e5-f9df-45a4-a2d6-c73d47104ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:06:20.912373 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee35e5-f9df-45a4-a2d6-c73d47104ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:06:20.912648 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee35e5-f9df-45a4-a2d6-c73d47104ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:06:20.914407 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-92ee35e5-f9df-45a4-a2d6-c73d47104ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:20.979676 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee35e5-f9df-45a4-a2d6-c73d47104ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:06:20.981566 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee35e5-f9df-45a4-a2d6-c73d47104ba4 service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:06:20.992495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1442/2872] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:06:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:06:52.574379 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cc0c8-8b9b-4c25-971b-11ffb38fae9e service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:06:52.576144 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f82cc0c8-8b9b-4c25-971b-11ffb38fae9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:52.690361 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cc0c8-8b9b-4c25-971b-11ffb38fae9e service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:06:52.696495 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb88de8-f126-4ce8-8866-1fd4dfd263d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:06:52.697154 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb88de8-f126-4ce8-8866-1fd4dfd263d8 service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:06:52.697434 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb88de8-f126-4ce8-8866-1fd4dfd263d8 service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:06:52.697711 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb88de8-f126-4ce8-8866-1fd4dfd263d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:06:52.700572 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-9fb88de8-f126-4ce8-8866-1fd4dfd263d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:52.731344 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb88de8-f126-4ce8-8866-1fd4dfd263d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:06:52.733590 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb88de8-f126-4ce8-8866-1fd4dfd263d8 service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:06:52.753214 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1431/2873] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:06:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:06:58.202208 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee35e5-f9df-45a4-a2d6-c73d47104ba4 service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:06:58.231464 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee35e5-f9df-45a4-a2d6-c73d47104ba4 service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:06:58.238937 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501022b-a0b6-441e-b654-db762f479ce2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:06:58.239610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501022b-a0b6-441e-b654-db762f479ce2 service nova] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:06:58.239892 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501022b-a0b6-441e-b654-db762f479ce2 service nova] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:06:58.240185 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501022b-a0b6-441e-b654-db762f479ce2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:06:58.242454 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6501022b-a0b6-441e-b654-db762f479ce2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:58.256369 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501022b-a0b6-441e-b654-db762f479ce2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:06:58.257970 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501022b-a0b6-441e-b654-db762f479ce2 service nova] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:06:58.271925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1443/2874] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:06:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 14:06:59.127397 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a70a3-08fb-4f2f-bf8c-fc6bdd725ef1 None None] REQUEST_METHOD: `POST` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:06:59.127949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a70a3-08fb-4f2f-bf8c-fc6bdd725ef1 None None] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:06:59.127949 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a70a3-08fb-4f2f-bf8c-fc6bdd725ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:06:59.146800 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.auth.core [None req-c35a70a3-08fb-4f2f-bf8c-fc6bdd725ef1 None None] MFA Rules not processed for user `598c323e8ed5455cbe4a568b360d8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=86362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 14:06:59.147837 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c35a70a3-08fb-4f2f-bf8c-fc6bdd725ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:59.206874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1432/2875] 10.222.0.79 () {58 vars in 967 bytes} [Fri May 13 14:06:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 14:06:59.265397 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501022b-a0b6-441e-b654-db762f479ce2 service nova] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:06:59.267779 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-6501022b-a0b6-441e-b654-db762f479ce2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:06:59.342276 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501022b-a0b6-441e-b654-db762f479ce2 service nova] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:06:59.347434 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a50f73-a3c3-43e6-90ed-759faf51244d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4139cfe906bd4c7d94226ab589577627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:06:59.348094 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a50f73-a3c3-43e6-90ed-759faf51244d service swift] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:06:59.348375 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a50f73-a3c3-43e6-90ed-759faf51244d service swift] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:06:59.348650 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a50f73-a3c3-43e6-90ed-759faf51244d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:06:59.357520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a50f73-a3c3-43e6-90ed-759faf51244d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:06:59.359294 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a50f73-a3c3-43e6-90ed-759faf51244d service swift] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:06:59.368547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1444/2876] 10.222.0.79 () {60 vars in 1384 bytes} [Fri May 13 14:06:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 14:07:06.975875 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a70a3-08fb-4f2f-bf8c-fc6bdd725ef1 None None] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:07:06.977453 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-c35a70a3-08fb-4f2f-bf8c-fc6bdd725ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:07:07.053226 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a70a3-08fb-4f2f-bf8c-fc6bdd725ef1 None None] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:07:07.058047 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:07:07.059214 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:07:07.059490 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:07:07.059762 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] PATH_INFO: `/v3/users/3bbfb51ebb5f4553a9ef7b70becf1a9a` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:07:07.061029 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bbfb51ebb5f4553a9ef7b70becf1a9a)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:07:07.062994 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:07:07.116856 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bbfb51ebb5f4553a9ef7b70becf1a9a'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.117499 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bbfb51ebb5f4553a9ef7b70becf1a9a'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.129123 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bbfb51ebb5f4553a9ef7b70becf1a9a'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.129810 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bbfb51ebb5f4553a9ef7b70becf1a9a'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.136715 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bbfb51ebb5f4553a9ef7b70becf1a9a'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.150561 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1433/2877] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:07:06 2022] DELETE /identity/v3/users/3bbfb51ebb5f4553a9ef7b70becf1a9a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:07:07.369262 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a50f73-a3c3-43e6-90ed-759faf51244d service swift] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:07:07.384504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a50f73-a3c3-43e6-90ed-759faf51244d service swift] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:07:07.445187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:07:07.446065 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:07:07.446350 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:07:07.446629 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] PATH_INFO: `/v3/projects/673435f01c47460bae84e4f31643f66d` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:07:07.447896 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=673435f01c47460bae84e4f31643f66d)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:07:07.449835 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:07:07.502828 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673435f01c47460bae84e4f31643f66d', 'request_id': 'req-70239698-a7a8-4085-a118-0a6c7e250c72'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.508270 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673435f01c47460bae84e4f31643f66d', 'request_id': 'req-70239698-a7a8-4085-a118-0a6c7e250c72'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.517090 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673435f01c47460bae84e4f31643f66d', 'request_id': 'req-70239698-a7a8-4085-a118-0a6c7e250c72'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.529096 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] The token cache is being invalidate because project 673435f01c47460bae84e4f31643f66d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:07:07.529331 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.529917 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.531547 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1445/2878] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:07:07 2022] DELETE /identity/v3/projects/673435f01c47460bae84e4f31643f66d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:07:07.536394 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:07:07.537945 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:07:07.551352 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218628d-79e4-4ae6-9c7b-6d25c6e4c9f0 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:07:07.612997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:07:07.613869 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:07:07.614178 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:07:07.614415 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] PATH_INFO: `/v3/users/ffb5f423e9cf48e2ac65191bb5d5c0b7` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:07:07.615631 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb5f423e9cf48e2ac65191bb5d5c0b7)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:07:07.617545 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:07:07.666091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb5f423e9cf48e2ac65191bb5d5c0b7'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.666825 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb5f423e9cf48e2ac65191bb5d5c0b7'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.679570 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb5f423e9cf48e2ac65191bb5d5c0b7'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.680497 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb5f423e9cf48e2ac65191bb5d5c0b7'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.687392 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb5f423e9cf48e2ac65191bb5d5c0b7'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:07.700628 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1434/2879] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:07:07 2022] DELETE /identity/v3/users/ffb5f423e9cf48e2ac65191bb5d5c0b7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:07:07.878977 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:07:07.880737 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:07:07.895134 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70239698-a7a8-4085-a118-0a6c7e250c72 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:07:07.954804 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:07:07.955682 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:07:07.955967 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:07:07.956247 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] PATH_INFO: `/v3/projects/089f0151b64d4854b0e0391b5aa74c8c` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:07:07.957478 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] RBAC: Authorizing `identity:delete_project(project_id=089f0151b64d4854b0e0391b5aa74c8c)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:07:07.959435 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:07:08.010606 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '089f0151b64d4854b0e0391b5aa74c8c', 'request_id': 'req-e0938694-7aec-49e5-b083-877d2eb4904c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:08.016766 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '089f0151b64d4854b0e0391b5aa74c8c', 'request_id': 'req-e0938694-7aec-49e5-b083-877d2eb4904c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:08.024129 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '089f0151b64d4854b0e0391b5aa74c8c', 'request_id': 'req-e0938694-7aec-49e5-b083-877d2eb4904c'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:08.035486 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] The token cache is being invalidate because project 089f0151b64d4854b0e0391b5aa74c8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:07:08.035764 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:08.036414 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:08.037941 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1446/2880] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:07:07 2022] DELETE /identity/v3/projects/089f0151b64d4854b0e0391b5aa74c8c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:07:37.996019 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:07:37.997547 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:07:38.102805 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca03da-f0da-447a-b00f-e590ff178cc8 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:07:38.148039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f9521-9f05-43d0-8799-7e94804da628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:07:38.148678 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f9521-9f05-43d0-8799-7e94804da628 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:07:38.148960 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f9521-9f05-43d0-8799-7e94804da628 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:07:38.149237 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f9521-9f05-43d0-8799-7e94804da628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:07:38.151030 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-536f9521-9f05-43d0-8799-7e94804da628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:07:38.266068 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f9521-9f05-43d0-8799-7e94804da628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:07:38.267560 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f9521-9f05-43d0-8799-7e94804da628 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:07:38.286979 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1435/2881] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:07:44.699154 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:07:44.700659 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:07:44.782364 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0938694-7aec-49e5-b083-877d2eb4904c admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:07:44.790142 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:07:44.791472 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:07:44.791757 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:07:44.792079 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] PATH_INFO: `/v3/users/a27e83f9f5fb4a0b99e7fac6ab90b3e4` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:07:44.793495 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a27e83f9f5fb4a0b99e7fac6ab90b3e4)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:07:44.795611 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:07:44.844179 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a27e83f9f5fb4a0b99e7fac6ab90b3e4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:44.856446 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a27e83f9f5fb4a0b99e7fac6ab90b3e4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:44.863105 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a27e83f9f5fb4a0b99e7fac6ab90b3e4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:44.863802 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a27e83f9f5fb4a0b99e7fac6ab90b3e4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:44.871565 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a27e83f9f5fb4a0b99e7fac6ab90b3e4'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:44.877728 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1447/2882] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:07:44 2022] DELETE /identity/v3/users/a27e83f9f5fb4a0b99e7fac6ab90b3e4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:07:45.065340 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f9521-9f05-43d0-8799-7e94804da628 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:07:45.079607 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f9521-9f05-43d0-8799-7e94804da628 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:07:45.133985 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:07:45.134871 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:07:45.135147 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:07:45.135416 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] PATH_INFO: `/v3/projects/fbbf18059349404c9ed7f53c5b38d2fd` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:07:45.136660 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbbf18059349404c9ed7f53c5b38d2fd)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:07:45.138565 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:07:45.185213 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbbf18059349404c9ed7f53c5b38d2fd', 'request_id': 'req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.191180 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbbf18059349404c9ed7f53c5b38d2fd', 'request_id': 'req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.197353 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbbf18059349404c9ed7f53c5b38d2fd', 'request_id': 'req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.210488 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] The token cache is being invalidate because project fbbf18059349404c9ed7f53c5b38d2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:07:45.210803 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.211436 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.213014 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1436/2883] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:07:45 2022] DELETE /identity/v3/projects/fbbf18059349404c9ed7f53c5b38d2fd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:07:45.217223 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:07:45.218654 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:07:45.232175 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442f874-2851-46ac-b3c7-5ea165b548e7 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:07:45.296056 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:07:45.296928 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:07:45.297215 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:07:45.297492 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] PATH_INFO: `/v3/users/830ee6c1c2014a2892272854bad61457` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:07:45.299187 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] RBAC: Authorizing `identity:delete_user(user_id=830ee6c1c2014a2892272854bad61457)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:07:45.301400 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:07:45.347394 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830ee6c1c2014a2892272854bad61457'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.358952 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830ee6c1c2014a2892272854bad61457'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.365573 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830ee6c1c2014a2892272854bad61457'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.366255 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830ee6c1c2014a2892272854bad61457'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.373632 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830ee6c1c2014a2892272854bad61457'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.379507 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1448/2884] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:07:45 2022] DELETE /identity/v3/users/830ee6c1c2014a2892272854bad61457 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:07:45.554208 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:07:45.555718 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:07:45.571511 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25c90b-cb72-4c7d-bd77-2f19c88a3b70 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:07:45.628964 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:07:45.629822 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:07:45.630100 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:07:45.630366 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] PATH_INFO: `/v3/projects/c54c962a109d4c52b606943ba49ade06` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:07:45.632039 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c54c962a109d4c52b606943ba49ade06)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:07:45.633486 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:07:45.678938 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c54c962a109d4c52b606943ba49ade06', 'request_id': 'req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.684853 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c54c962a109d4c52b606943ba49ade06', 'request_id': 'req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.690053 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c54c962a109d4c52b606943ba49ade06', 'request_id': 'req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.702141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] The token cache is being invalidate because project c54c962a109d4c52b606943ba49ade06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:07:45.702426 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.703084 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:07:45.704675 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1437/2885] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:07:45 2022] DELETE /identity/v3/projects/c54c962a109d4c52b606943ba49ade06 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:08:34.296360 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:08:34.297803 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:08:34.374382 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5385c-ad27-40cc-acc0-376f8fbd976f admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:08:34.437520 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d663dd-9506-417d-ae3f-b87a9d6bf9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:08:34.438234 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d663dd-9506-417d-ae3f-b87a9d6bf9fd service neutron] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:08:34.438725 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d663dd-9506-417d-ae3f-b87a9d6bf9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:08:34.439084 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d663dd-9506-417d-ae3f-b87a9d6bf9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:08:34.441518 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-42d663dd-9506-417d-ae3f-b87a9d6bf9fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:08:34.453940 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d663dd-9506-417d-ae3f-b87a9d6bf9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:08:34.455647 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d663dd-9506-417d-ae3f-b87a9d6bf9fd service neutron] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:08:34.464542 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1449/2886] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:08:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:08:34.539091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:08:34.541107 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:08:34.553527 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3aee-1760-47f7-b078-ba8fdd6e2e0b admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:08:34.557091 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b2843-d35e-4cdc-8a4b-267d7e7fb503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f784cc461ab48ad92140680758a3704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:08:34.557721 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b2843-d35e-4cdc-8a4b-267d7e7fb503 service neutron] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:08:34.558001 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b2843-d35e-4cdc-8a4b-267d7e7fb503 service neutron] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:08:34.558266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b2843-d35e-4cdc-8a4b-267d7e7fb503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:08:34.560072 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-500b2843-d35e-4cdc-8a4b-267d7e7fb503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:08:34.635673 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b2843-d35e-4cdc-8a4b-267d7e7fb503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:08:34.637141 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b2843-d35e-4cdc-8a4b-267d7e7fb503 service neutron] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:08:34.648723 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1438/2887] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:08:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:08:34.720257 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d663dd-9506-417d-ae3f-b87a9d6bf9fd service neutron] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:08:34.721655 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-42d663dd-9506-417d-ae3f-b87a9d6bf9fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:08:34.789476 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d663dd-9506-417d-ae3f-b87a9d6bf9fd service neutron] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:08:34.795553 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f172013-5653-4362-b09e-4fcf07ff6ce6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ae9a6f1ac304e21b8d3115f2a0baf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:08:34.796207 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f172013-5653-4362-b09e-4fcf07ff6ce6 service placement] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:08:34.796485 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f172013-5653-4362-b09e-4fcf07ff6ce6 service placement] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:08:34.796754 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f172013-5653-4362-b09e-4fcf07ff6ce6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:08:34.798942 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2f172013-5653-4362-b09e-4fcf07ff6ce6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:08:34.811162 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f172013-5653-4362-b09e-4fcf07ff6ce6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:08:34.812925 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f172013-5653-4362-b09e-4fcf07ff6ce6 service placement] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:08:34.823504 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1450/2888] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:08:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:08:37.094254 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b2843-d35e-4cdc-8a4b-267d7e7fb503 service neutron] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:08:37.096044 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-500b2843-d35e-4cdc-8a4b-267d7e7fb503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:08:37.111142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b2843-d35e-4cdc-8a4b-267d7e7fb503 service neutron] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:08:37.116881 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ed467-dc87-40e0-860b-39141c3e850f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ad0dd9d265425087be2be8d245b7fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:08:37.117566 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ed467-dc87-40e0-860b-39141c3e850f service nova] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:08:37.117843 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ed467-dc87-40e0-860b-39141c3e850f service nova] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:08:37.118115 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ed467-dc87-40e0-860b-39141c3e850f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:08:37.120348 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-960ed467-dc87-40e0-860b-39141c3e850f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:08:37.133639 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ed467-dc87-40e0-860b-39141c3e850f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:08:37.136266 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ed467-dc87-40e0-860b-39141c3e850f service nova] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:08:37.147619 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1439/2889] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:08:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:09:50.791357 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f172013-5653-4362-b09e-4fcf07ff6ce6 service placement] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:09:50.792953 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-2f172013-5653-4362-b09e-4fcf07ff6ce6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:09:50.830543 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f172013-5653-4362-b09e-4fcf07ff6ce6 service placement] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:09:50.835517 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e49886-728e-40d3-b18f-e1245e311f93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ae9a6f1ac304e21b8d3115f2a0baf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:09:50.836151 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e49886-728e-40d3-b18f-e1245e311f93 service placement] REQUEST_METHOD: `GET` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:09:50.836428 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e49886-728e-40d3-b18f-e1245e311f93 service placement] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:09:50.836698 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e49886-728e-40d3-b18f-e1245e311f93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:09:50.838489 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-d0e49886-728e-40d3-b18f-e1245e311f93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:09:50.849972 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e49886-728e-40d3-b18f-e1245e311f93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:09:50.851610 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e49886-728e-40d3-b18f-e1245e311f93 service placement] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:09:50.862073 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1451/2890] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:09:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 14:10:10.089241 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ed467-dc87-40e0-860b-39141c3e850f service nova] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:10:10.090745 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-960ed467-dc87-40e0-860b-39141c3e850f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:10:10.154644 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ed467-dc87-40e0-860b-39141c3e850f service nova] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:10:10.159738 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:10:10.160602 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:10:10.160874 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:10:10.161142 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] PATH_INFO: `/v3/users/4ac7988aa0e74fc6b56269d975270d48` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:10:10.162313 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac7988aa0e74fc6b56269d975270d48)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:10:10.164260 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:10:10.213606 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac7988aa0e74fc6b56269d975270d48'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.214315 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac7988aa0e74fc6b56269d975270d48'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.228777 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac7988aa0e74fc6b56269d975270d48'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.229513 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac7988aa0e74fc6b56269d975270d48'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.236966 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac7988aa0e74fc6b56269d975270d48'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.249840 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1440/2891] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:10:10 2022] DELETE /identity/v3/users/4ac7988aa0e74fc6b56269d975270d48 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:10:10.421167 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e49886-728e-40d3-b18f-e1245e311f93 service placement] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:10:10.432245 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e49886-728e-40d3-b18f-e1245e311f93 service placement] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:10:10.489114 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:10:10.489978 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:10:10.490260 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:10:10.490537 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] PATH_INFO: `/v3/projects/54b153696d2b445098e70dce631b13b6` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:10:10.491760 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54b153696d2b445098e70dce631b13b6)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:10:10.493672 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:10:10.537934 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54b153696d2b445098e70dce631b13b6', 'request_id': 'req-16bf0267-5166-4952-a67d-575c9ab8f990'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.542975 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54b153696d2b445098e70dce631b13b6', 'request_id': 'req-16bf0267-5166-4952-a67d-575c9ab8f990'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.550193 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54b153696d2b445098e70dce631b13b6', 'request_id': 'req-16bf0267-5166-4952-a67d-575c9ab8f990'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.560809 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] The token cache is being invalidate because project 54b153696d2b445098e70dce631b13b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:10:10.561120 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.561744 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.564302 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1452/2892] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:10:10 2022] DELETE /identity/v3/projects/54b153696d2b445098e70dce631b13b6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:10:10.568731 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:10:10.570223 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:10:10.583997 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3cbc3-4d43-46c1-a5f1-c6ce515eea89 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:10:10.638440 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:10:10.639332 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:10:10.639611 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:10:10.639879 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] PATH_INFO: `/v3/users/029e07de814546e59202df728df9a823` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:10:10.641558 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=029e07de814546e59202df728df9a823)` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:10:10.643508 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:10:10.688819 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '029e07de814546e59202df728df9a823'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.689702 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '029e07de814546e59202df728df9a823'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.701197 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '029e07de814546e59202df728df9a823'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.701839 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '029e07de814546e59202df728df9a823'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.708920 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.notifications [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '029e07de814546e59202df728df9a823'} {{(pid=86362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:10.722914 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1441/2893] 10.222.0.79 () {58 vars in 1223 bytes} [Fri May 13 14:10:10 2022] DELETE /identity/v3/users/029e07de814546e59202df728df9a823 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 14:10:10.898361 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] Authenticating user token {{(pid=86363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:10:10.900697 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.fernet_utils [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:10:10.915323 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf0267-5166-4952-a67d-575c9ab8f990 admin admin] Validating token access rules against request {{(pid=86363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:10:10.972017 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0034eb1f585466fae466bc41ea6f933', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '584c523555e7486e977f27ddcc51047b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:10:10.972890 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:10:10.973168 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] SCRIPT_NAME: `/identity` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:10:10.973441 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] PATH_INFO: `/v3/projects/25aece37f16d46259f10198b17d91fa1` {{(pid=86363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:10:10.974636 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=25aece37f16d46259f10198b17d91fa1)` {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:10:10.976577 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] RBAC: Authorization granted {{(pid=86363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:10:11.022548 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25aece37f16d46259f10198b17d91fa1', 'request_id': 'req-16469b08-3dda-4b30-bc63-4f3968970aac'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:11.027848 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25aece37f16d46259f10198b17d91fa1', 'request_id': 'req-16469b08-3dda-4b30-bc63-4f3968970aac'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:11.035086 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25aece37f16d46259f10198b17d91fa1', 'request_id': 'req-16469b08-3dda-4b30-bc63-4f3968970aac'} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:11.045743 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] The token cache is being invalidate because project 25aece37f16d46259f10198b17d91fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 14:10:11.046026 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:11.046748 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: DEBUG keystone.notifications [None req-16469b08-3dda-4b30-bc63-4f3968970aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 14:10:11.048661 dv-cinder-841753-1-iscsi devstack@keystone.service[86363]: [pid: 86363|app: 0|req: 1453/2894] 10.222.0.79 () {58 vars in 1232 bytes} [Fri May 13 14:10:10 2022] DELETE /identity/v3/projects/25aece37f16d46259f10198b17d91fa1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 14:14:53.795251 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] Authenticating user token {{(pid=86362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 14:14:53.796923 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:14:53.863838 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d8b36-2884-4b78-9a65-c4e31fa42064 admin admin] Validating token access rules against request {{(pid=86362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 14:14:53.922550 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ac7aa-04b2-432f-a571-246f38c832e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ae9a6f1ac304e21b8d3115f2a0baf88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129cd7365a3f45a1ba4c0308a6b1cacd', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 14:14:53.923279 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ac7aa-04b2-432f-a571-246f38c832e2 service placement] REQUEST_METHOD: `GET` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 14:14:53.923553 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ac7aa-04b2-432f-a571-246f38c832e2 service placement] SCRIPT_NAME: `/identity` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 14:14:53.923827 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ac7aa-04b2-432f-a571-246f38c832e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 14:14:53.925592 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.fernet_utils [None req-954ac7aa-04b2-432f-a571-246f38c832e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 14:14:53.937557 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ac7aa-04b2-432f-a571-246f38c832e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 14:14:53.939229 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ac7aa-04b2-432f-a571-246f38c832e2 service placement] RBAC: Authorization granted {{(pid=86362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 14:14:53.950564 dv-cinder-841753-1-iscsi devstack@keystone.service[86362]: [pid: 86362|app: 0|req: 1442/2895] 10.222.0.79 () {60 vars in 1365 bytes} [Fri May 13 14:14:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)